[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502156971023-7698-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 02:35:57 dist-test-slave-mn4r krb5kdc[7706](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 02:35:57 dist-test-slave-mn4r krb5kdc[7706](info): set up 2 sockets
Aug 12 02:35:57 dist-test-slave-mn4r krb5kdc[7706](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260812 02:35:58.289465 7698 mini_kdc.cc:121] Time spent starting KDC: real 1.230s user 0.004s sys 0.016s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502156971023-7698-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502156971023-7698-0/krb5kdc/test-user.keytab.
Aug 12 02:35:58 dist-test-slave-mn4r krb5kdc[7706](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502158, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T02:35:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T02:35:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.132.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
I20260812 02:35:58.508795 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502156971023-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502156971023-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502156971023-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:46247
--webserver_interface=127.7.132.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:43769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.132.190:46247
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:35:58.860857 7722 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:35:58.861173 7722 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:35:58.861339 7722 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:35:58.868709 7722 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:35:58.868821 7722 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:35:58.868942 7722 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:35:58.869048 7722 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:35:58.869138 7722 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:35:58.879395 7722 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:43769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502156971023-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502156971023-7698-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.132.190:46247
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:46247
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.132.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502156971023-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:35:58.880865 7722 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:35:58.882390 7722 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:35:58.895272 7735 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:35:58.895355 7729 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:35:58.895375 7731 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:35:58.895844 7722 server_base.cc:1034] running on GCE node
I20260812 02:35:58.896920 7722 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:35:58.898543 7722 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:35:58.899796 7722 hybrid_clock.cc:648] HybridClock initialized: now 1786502158899754 us; error 109 us; skew 500 ppm
I20260812 02:35:58.900167 7722 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:35:58 dist-test-slave-mn4r krb5kdc[7706](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502158, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:35:58.903671 7722 init.cc:377] Logged in from keytab as kudu/127.7.132.190@KRBTEST.COM (short username kudu)
I20260812 02:35:58.905349 7722 webserver.cc:460] Webserver started at http://127.7.132.190:46211/ using document root <none> and password file <none>
I20260812 02:35:58.906160 7722 fs_manager.cc:359] Metadata directory not provided
I20260812 02:35:58.906313 7722 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:35:58.906658 7722 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:35:58.910212 7722 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502156971023-7698-0/minicluster-data/master-0/data/instance:
uuid: "b7118285e37344728de50982e908226e"
format_stamp: "Formatted at 2026-08-12 02:35:58 on dist-test-slave-mn4r"
server_key: "00b196ad719120987e663d1c777ea2fe"
server_key_iv: "43ae075ea73c4cfcef405205106fe505"
server_key_version: "encryptionkey@0"
I20260812 02:35:58.911150 7722 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502156971023-7698-0/minicluster-data/master-0/wal/instance:
uuid: "b7118285e37344728de50982e908226e"
format_stamp: "Formatted at 2026-08-12 02:35:58 on dist-test-slave-mn4r"
server_key: "00b196ad719120987e663d1c777ea2fe"
server_key_iv: "43ae075ea73c4cfcef405205106fe505"
server_key_version: "encryptionkey@0"
I20260812 02:35:58.916855 7722 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.001s
I20260812 02:35:58.920706 7743 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:35:58.922333 7722 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260812 02:35:58.922526 7722 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502156971023-7698-0/minicluster-data/master-0/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502156971023-7698-0/minicluster-data/master-0/wal
uuid: "b7118285e37344728de50982e908226e"
format_stamp: "Formatted at 2026-08-12 02:35:58 on dist-test-slave-mn4r"
server_key: "00b196ad719120987e663d1c777ea2fe"
server_key_iv: "43ae075ea73c4cfcef405205106fe505"
server_key_version: "encryptionkey@0"
I20260812 02:35:58.922747 7722 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502156971023-7698-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502156971023-7698-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502156971023-7698-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:35:58.938448 7722 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:35:58.942988 7722 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:35:58.943320 7722 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:35:58.963195 7722 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.190:46247
I20260812 02:35:58.963217 7800 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.190:46247 every 8 connection(s)
I20260812 02:35:58.964710 7722 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
I20260812 02:35:58.968820 7801 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:35:58.971138 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 7722
I20260812 02:35:58.971590 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502156971023-7698-0/minicluster-data/master-0/wal/instance
I20260812 02:35:58.974233 7698 external_mini_cluster.cc:1428] Setting key 2a9bbc875bbb0ab2544c17365d5488d4
I20260812 02:35:58.983707 7801 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b7118285e37344728de50982e908226e: Bootstrap starting.
I20260812 02:35:58.987303 7801 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b7118285e37344728de50982e908226e: Neither blocks nor log segments found. Creating new log.
I20260812 02:35:58.988538 7801 log.cc:824] T 00000000000000000000000000000000 P b7118285e37344728de50982e908226e: Log is configured to *not* fsync() on all Append() calls
I20260812 02:35:58.992614 7801 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b7118285e37344728de50982e908226e: No bootstrap required, opened a new log
I20260812 02:35:59.002321 7801 raft_consensus.cc:348] T 00000000000000000000000000000000 P b7118285e37344728de50982e908226e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7118285e37344728de50982e908226e" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 46247 } }
I20260812 02:35:59.002735 7801 raft_consensus.cc:374] T 00000000000000000000000000000000 P b7118285e37344728de50982e908226e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:35:59.002895 7801 raft_consensus.cc:729] T 00000000000000000000000000000000 P b7118285e37344728de50982e908226e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7118285e37344728de50982e908226e, State: Initialized, Role: FOLLOWER
I20260812 02:35:59.003634 7801 consensus_queue.cc:260] T 00000000000000000000000000000000 P b7118285e37344728de50982e908226e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7118285e37344728de50982e908226e" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 46247 } }
I20260812 02:35:59.003909 7801 raft_consensus.cc:388] T 00000000000000000000000000000000 P b7118285e37344728de50982e908226e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 02:35:59.004076 7801 raft_consensus.cc:482] T 00000000000000000000000000000000 P b7118285e37344728de50982e908226e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 02:35:59.004256 7801 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b7118285e37344728de50982e908226e [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:35:59.006762 7801 raft_consensus.cc:504] T 00000000000000000000000000000000 P b7118285e37344728de50982e908226e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7118285e37344728de50982e908226e" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 46247 } }
Aug 12 02:35:59 dist-test-slave-mn4r krb5kdc[7706](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502158, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 02:35:59.007330 7801 leader_election.cc:302] T 00000000000000000000000000000000 P b7118285e37344728de50982e908226e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b7118285e37344728de50982e908226e; no voters:
I20260812 02:35:59.008004 7801 leader_election.cc:288] T 00000000000000000000000000000000 P b7118285e37344728de50982e908226e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 02:35:59.008232 7808 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b7118285e37344728de50982e908226e [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:35:59.009334 7808 raft_consensus.cc:686] T 00000000000000000000000000000000 P b7118285e37344728de50982e908226e [term 1 LEADER]: Becoming Leader. State: Replica: b7118285e37344728de50982e908226e, State: Running, Role: LEADER
I20260812 02:35:59.010017 7808 consensus_queue.cc:237] T 00000000000000000000000000000000 P b7118285e37344728de50982e908226e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7118285e37344728de50982e908226e" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 46247 } }
I20260812 02:35:59.010586 7801 sys_catalog.cc:564] T 00000000000000000000000000000000 P b7118285e37344728de50982e908226e [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:35:59.013182 7810 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7118285e37344728de50982e908226e [sys.catalog]: SysCatalogTable state changed. Reason: New leader b7118285e37344728de50982e908226e. Latest consensus state: current_term: 1 leader_uuid: "b7118285e37344728de50982e908226e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7118285e37344728de50982e908226e" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 46247 } } }
I20260812 02:35:59.014977 7810 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7118285e37344728de50982e908226e [sys.catalog]: This master's current role is: LEADER
I20260812 02:35:59.014904 7809 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7118285e37344728de50982e908226e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b7118285e37344728de50982e908226e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7118285e37344728de50982e908226e" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 46247 } } }
I20260812 02:35:59.015245 7809 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7118285e37344728de50982e908226e [sys.catalog]: This master's current role is: LEADER
I20260812 02:35:59.018373 7813 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:35:59.020767 7804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:35:58.979294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47282 (local address 127.7.132.190:46247)
0812 02:35:58.980484 (+ 1190us) server_negotiation.cc:206] Beginning negotiation
0812 02:35:58.980504 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0812 02:35:58.981924 (+ 1420us) server_negotiation.cc:407] Connection header received
0812 02:35:58.992232 (+ 10308us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:35:58.992262 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:35:58.992845 (+ 583us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:35:58.993445 (+ 600us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:35:58.996423 (+ 2978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:35:58.998762 (+ 2339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:35:59.000570 (+ 1808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:35:59.001732 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:35:59.001975 (+ 243us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:35:59.008643 (+ 6668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:35:59.008659 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:35:59.008687 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:35:59.008765 (+ 78us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:35:59.013784 (+ 5019us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:35:59.014854 (+ 1070us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:35:59.014865 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:35:59.014876 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:35:59.015135 (+ 259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:35:59.015832 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:35:59.015839 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:35:59.015845 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:35:59.016851 (+ 1006us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:35:59.018212 (+ 1361us) server_negotiation.cc:1035] Waiting for connection context
0812 02:35:59.018525 (+ 313us) server_negotiation.cc:299] Negotiation successful
0812 02:35:59.018973 (+ 448us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":883,"thread_start_us":589,"threads_started":1}
I20260812 02:35:59.024251 7813 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 02:35:59.034364 7813 catalog_manager.cc:1347] Generated new cluster ID: f4373ae5100540e7ac6e09b23a0b26d5
I20260812 02:35:59.034513 7813 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 02:35:59.050129 7813 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 02:35:59.052066 7813 catalog_manager.cc:1504] Loading token signing keys...
I20260812 02:35:59.068540 7813 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b7118285e37344728de50982e908226e: Generated new TSK 0
I20260812 02:35:59.069535 7813 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.132.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.129.keytab.
Entry for principal kudu/127.7.132.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.129.keytab.
I20260812 02:35:59.125993 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502156971023-7698-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502156971023-7698-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502156971023-7698-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502156971023-7698-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.129:0
--local_ip_for_outbound_sockets=127.7.132.129
--webserver_interface=127.7.132.129
--webserver_port=0
--tserver_master_addrs=127.7.132.190:46247
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.129.keytab
--principal=kudu/127.7.132.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:43769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:35:59.493331 7837 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:35:59.493726 7837 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:35:59.493924 7837 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:35:59.501753 7837 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:35:59.501988 7837 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:35:59.502218 7837 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.129
I20260812 02:35:59.513214 7837 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:43769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502156971023-7698-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502156971023-7698-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.129.keytab
--principal=kudu/127.7.132.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502156971023-7698-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.132.129
--webserver_port=0
--tserver_master_addrs=127.7.132.190:46247
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.129
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502156971023-7698-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:35:59.514874 7837 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:35:59.516440 7837 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:35:59.531200 7843 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:35:59.531149 7846 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:35:59.531141 7844 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:35:59.532287 7837 server_base.cc:1034] running on GCE node
I20260812 02:35:59.533036 7837 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:35:59.534072 7837 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:35:59.535355 7837 hybrid_clock.cc:648] HybridClock initialized: now 1786502159535304 us; error 41 us; skew 500 ppm
I20260812 02:35:59.535732 7837 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:35:59 dist-test-slave-mn4r krb5kdc[7706](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502159, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:35:59.539232 7837 init.cc:377] Logged in from keytab as kudu/127.7.132.129@KRBTEST.COM (short username kudu)
I20260812 02:35:59.540894 7837 webserver.cc:460] Webserver started at http://127.7.132.129:43607/ using document root <none> and password file <none>
I20260812 02:35:59.541652 7837 fs_manager.cc:359] Metadata directory not provided
I20260812 02:35:59.541811 7837 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:35:59.542246 7837 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:35:59.545351 7837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502156971023-7698-0/minicluster-data/ts-0/data/instance:
uuid: "ab44c6322f624b11bd6bf11a5e96de13"
format_stamp: "Formatted at 2026-08-12 02:35:59 on dist-test-slave-mn4r"
server_key: "20e8bdeca7a1dcc644302aa5e51b16f4"
server_key_iv: "4151e9ac193b8041c3df2ba8cb602bb7"
server_key_version: "encryptionkey@0"
I20260812 02:35:59.546355 7837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502156971023-7698-0/minicluster-data/ts-0/wal/instance:
uuid: "ab44c6322f624b11bd6bf11a5e96de13"
format_stamp: "Formatted at 2026-08-12 02:35:59 on dist-test-slave-mn4r"
server_key: "20e8bdeca7a1dcc644302aa5e51b16f4"
server_key_iv: "4151e9ac193b8041c3df2ba8cb602bb7"
server_key_version: "encryptionkey@0"
I20260812 02:35:59.551908 7837 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 02:35:59.555682 7856 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:35:59.557322 7837 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.003s
I20260812 02:35:59.557538 7837 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502156971023-7698-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502156971023-7698-0/minicluster-data/ts-0/wal
uuid: "ab44c6322f624b11bd6bf11a5e96de13"
format_stamp: "Formatted at 2026-08-12 02:35:59 on dist-test-slave-mn4r"
server_key: "20e8bdeca7a1dcc644302aa5e51b16f4"
server_key_iv: "4151e9ac193b8041c3df2ba8cb602bb7"
server_key_version: "encryptionkey@0"
I20260812 02:35:59.557803 7837 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502156971023-7698-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502156971023-7698-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502156971023-7698-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:35:59.589071 7837 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:35:59.593689 7837 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:35:59.594026 7837 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:35:59.595165 7837 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:35:59.596661 7837 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:35:59.596769 7837 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:35:59.596920 7837 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:35:59.597004 7837 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:35:59.632668 7837 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.129:35513
I20260812 02:35:59.632722 7988 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.129:35513 every 8 connection(s)
I20260812 02:35:59.634121 7837 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502156971023-7698-0/minicluster-data/ts-0/data/info.pb
I20260812 02:35:59.639400 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 7837
I20260812 02:35:59.639675 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502156971023-7698-0/minicluster-data/ts-0/wal/instance
I20260812 02:35:59.640322 7698 external_mini_cluster.cc:1428] Setting key 0ac297c68d8bf6ec6e1a008fcf313cde
Aug 12 02:35:59 dist-test-slave-mn4r krb5kdc[7706](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502159, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.129@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 02:35:59.657950 7991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:35:59.636921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:40863 (local address 127.7.132.190:46247)
0812 02:35:59.637414 (+ 493us) server_negotiation.cc:206] Beginning negotiation
0812 02:35:59.637420 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:35:59.638654 (+ 1234us) server_negotiation.cc:407] Connection header received
0812 02:35:59.640837 (+ 2183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:35:59.640843 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:35:59.640938 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:35:59.641089 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:35:59.643352 (+ 2263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:35:59.644675 (+ 1323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:35:59.646547 (+ 1872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:35:59.647356 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:35:59.647534 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:35:59.651378 (+ 3844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:35:59.651384 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:35:59.651387 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:35:59.651426 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:35:59.654432 (+ 3006us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:35:59.655296 (+ 864us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:35:59.655302 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:35:59.655305 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:35:59.655437 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:35:59.655943 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:35:59.655951 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:35:59.655954 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:35:59.656317 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:35:59.656495 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0812 02:35:59.657305 (+ 810us) server_negotiation.cc:299] Negotiation successful
0812 02:35:59.657552 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":238,"threads_started":1}
I20260812 02:35:59.659152 7992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:35:59.637658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:46247 (local address 127.7.132.129:40863)
0812 02:35:59.638461 (+ 803us) negotiation.cc:107] Waiting for socket to connect
0812 02:35:59.638516 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0812 02:35:59.640509 (+ 1993us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:35:59.641429 (+ 920us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:35:59.641439 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:35:59.642313 (+ 874us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:35:59.643137 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:35:59.643158 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:35:59.644900 (+ 1742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:35:59.644904 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:35:59.646383 (+ 1479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:35:59.646398 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:35:59.647611 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:35:59.647616 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:35:59.647827 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:35:59.648608 (+ 781us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:35:59.648632 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:35:59.651085 (+ 2453us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:35:59.654615 (+ 3530us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:35:59.654626 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:35:59.654653 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:35:59.655139 (+ 486us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:35:59.655564 (+ 425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:35:59.655569 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:35:59.655573 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:35:59.655798 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:35:59.656469 (+ 671us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:35:59.656477 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:35:59.656877 (+ 400us) client_negotiation.cc:769] Sending connection context
0812 02:35:59.657174 (+ 297us) client_negotiation.cc:240] Negotiation successful
0812 02:35:59.657797 (+ 623us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":548,"thread_start_us":363,"threads_started":1}
WARNING: no policy specified for kudu/127.7.132.130@KRBTEST.COM; defaulting to no policy
I20260812 02:35:59.661160 7989 heartbeater.cc:347] Connected to a master server at 127.7.132.190:46247
I20260812 02:35:59.661608 7989 heartbeater.cc:464] Registering TS with master...
I20260812 02:35:59.662655 7989 heartbeater.cc:511] Master 127.7.132.190:46247 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.130@KRBTEST.COM" created.
I20260812 02:35:59.665257 7762 ts_manager.cc:194] Registered new tserver with Master: ab44c6322f624b11bd6bf11a5e96de13 (127.7.132.129:35513)
I20260812 02:35:59.666914 7762 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.129@KRBTEST.COM'} at 127.7.132.129:40863
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.130.keytab.
Entry for principal kudu/127.7.132.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.130.keytab.
I20260812 02:35:59.689888 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502156971023-7698-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502156971023-7698-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502156971023-7698-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502156971023-7698-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.130:0
--local_ip_for_outbound_sockets=127.7.132.130
--webserver_interface=127.7.132.130
--webserver_port=0
--tserver_master_addrs=127.7.132.190:46247
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.130.keytab
--principal=kudu/127.7.132.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:43769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:36:00.044924 7995 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:36:00.045269 7995 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:36:00.045382 7995 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:36:00.052949 7995 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:36:00.053087 7995 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:36:00.053213 7995 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.130
I20260812 02:36:00.063948 7995 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:43769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502156971023-7698-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502156971023-7698-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.130.keytab
--principal=kudu/127.7.132.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502156971023-7698-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.132.130
--webserver_port=0
--tserver_master_addrs=127.7.132.190:46247
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.130
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502156971023-7698-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:36:00.065363 7995 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:36:00.066874 7995 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:36:00.081385 8003 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:36:00.081353 8004 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:36:00.081434 8007 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:36:00.082377 7995 server_base.cc:1034] running on GCE node
I20260812 02:36:00.083137 7995 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:36:00.084234 7995 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:36:00.085453 7995 hybrid_clock.cc:648] HybridClock initialized: now 1786502160085375 us; error 64 us; skew 500 ppm
I20260812 02:36:00.085817 7995 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:36:00 dist-test-slave-mn4r krb5kdc[7706](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502160, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:36:00.089527 7995 init.cc:377] Logged in from keytab as kudu/127.7.132.130@KRBTEST.COM (short username kudu)
I20260812 02:36:00.091207 7995 webserver.cc:460] Webserver started at http://127.7.132.130:35123/ using document root <none> and password file <none>
I20260812 02:36:00.092065 7995 fs_manager.cc:359] Metadata directory not provided
I20260812 02:36:00.092176 7995 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:36:00.092545 7995 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:36:00.095795 7995 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502156971023-7698-0/minicluster-data/ts-1/data/instance:
uuid: "cf1cac3cafe642c1b27d5b80dcfb1c4f"
format_stamp: "Formatted at 2026-08-12 02:36:00 on dist-test-slave-mn4r"
server_key: "f5da7e5cfbb052c1a2e72a0815a04760"
server_key_iv: "8f71d583a89470a66da8a6aeb92a4b08"
server_key_version: "encryptionkey@0"
I20260812 02:36:00.096632 7995 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502156971023-7698-0/minicluster-data/ts-1/wal/instance:
uuid: "cf1cac3cafe642c1b27d5b80dcfb1c4f"
format_stamp: "Formatted at 2026-08-12 02:36:00 on dist-test-slave-mn4r"
server_key: "f5da7e5cfbb052c1a2e72a0815a04760"
server_key_iv: "8f71d583a89470a66da8a6aeb92a4b08"
server_key_version: "encryptionkey@0"
I20260812 02:36:00.102671 7995 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.004s
I20260812 02:36:00.106673 8016 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:36:00.108333 7995 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 02:36:00.108516 7995 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502156971023-7698-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502156971023-7698-0/minicluster-data/ts-1/wal
uuid: "cf1cac3cafe642c1b27d5b80dcfb1c4f"
format_stamp: "Formatted at 2026-08-12 02:36:00 on dist-test-slave-mn4r"
server_key: "f5da7e5cfbb052c1a2e72a0815a04760"
server_key_iv: "8f71d583a89470a66da8a6aeb92a4b08"
server_key_version: "encryptionkey@0"
I20260812 02:36:00.108729 7995 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502156971023-7698-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502156971023-7698-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502156971023-7698-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:36:00.136003 7995 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:36:00.140538 7995 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:36:00.140802 7995 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:36:00.141860 7995 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:36:00.143342 7995 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:36:00.143419 7995 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:36:00.143515 7995 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:36:00.143561 7995 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:36:00.179951 7995 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.130:44211
I20260812 02:36:00.180065 8155 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.130:44211 every 8 connection(s)
I20260812 02:36:00.181396 7995 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502156971023-7698-0/minicluster-data/ts-1/data/info.pb
I20260812 02:36:00.182654 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 7995
I20260812 02:36:00.182868 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502156971023-7698-0/minicluster-data/ts-1/wal/instance
I20260812 02:36:00.183423 7698 external_mini_cluster.cc:1428] Setting key dff05476d19a78eb88cd00223f8a6d4a
Aug 12 02:36:00 dist-test-slave-mn4r krb5kdc[7706](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502160, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.130@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.132.131@KRBTEST.COM; defaulting to no policy
I20260812 02:36:00.208559 8158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:00.184545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:50717 (local address 127.7.132.190:46247)
0812 02:36:00.185050 (+ 505us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:00.185056 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:00.186592 (+ 1536us) server_negotiation.cc:407] Connection header received
0812 02:36:00.189767 (+ 3175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:00.189772 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:00.189864 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:00.189969 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:00.192360 (+ 2391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:00.193699 (+ 1339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:00.195442 (+ 1743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:00.196347 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:00.196524 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:00.200547 (+ 4023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:00.200558 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:00.200565 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:00.200620 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:00.204947 (+ 4327us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:00.205891 (+ 944us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:00.205897 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:00.205901 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:00.206122 (+ 221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:00.206698 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:00.206703 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:00.206705 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:00.207039 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:00.207170 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:00.208018 (+ 848us) server_negotiation.cc:299] Negotiation successful
0812 02:36:00.208217 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":379,"thread_start_us":258,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.131@KRBTEST.COM" created.
I20260812 02:36:00.209810 8159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:00.185332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:46247 (local address 127.7.132.130:50717)
0812 02:36:00.186385 (+ 1053us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:00.186453 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:00.189381 (+ 2928us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:00.190318 (+ 937us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:00.190334 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:00.191216 (+ 882us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:36:00.192133 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:00.192163 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:00.193901 (+ 1738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:00.193905 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:00.195272 (+ 1367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:00.195286 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:00.196600 (+ 1314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:00.196605 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:00.196823 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:00.197602 (+ 779us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:36:00.197627 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:36:00.200202 (+ 2575us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:36:00.205151 (+ 4949us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:00.205160 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:00.205176 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:00.205715 (+ 539us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:00.206291 (+ 576us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:00.206295 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:00.206299 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:00.206538 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:00.207180 (+ 642us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:36:00.207186 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:36:00.207621 (+ 435us) client_negotiation.cc:769] Sending connection context
0812 02:36:00.207879 (+ 258us) client_negotiation.cc:240] Negotiation successful
0812 02:36:00.208344 (+ 465us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":735,"thread_start_us":439,"threads_started":1}
I20260812 02:36:00.211779 8156 heartbeater.cc:347] Connected to a master server at 127.7.132.190:46247
I20260812 02:36:00.212356 8156 heartbeater.cc:464] Registering TS with master...
I20260812 02:36:00.213387 8156 heartbeater.cc:511] Master 127.7.132.190:46247 requested a full tablet report, sending...
I20260812 02:36:00.215166 7762 ts_manager.cc:194] Registered new tserver with Master: cf1cac3cafe642c1b27d5b80dcfb1c4f (127.7.132.130:44211)
I20260812 02:36:00.216053 7762 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.130@KRBTEST.COM'} at 127.7.132.130:50717
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.131.keytab.
Entry for principal kudu/127.7.132.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.131.keytab.
I20260812 02:36:00.238655 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502156971023-7698-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502156971023-7698-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502156971023-7698-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502156971023-7698-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.131:0
--local_ip_for_outbound_sockets=127.7.132.131
--webserver_interface=127.7.132.131
--webserver_port=0
--tserver_master_addrs=127.7.132.190:46247
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.131.keytab
--principal=kudu/127.7.132.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:43769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:36:00.599098 8165 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:36:00.599395 8165 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:36:00.599603 8165 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:36:00.606925 8165 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:36:00.607035 8165 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:36:00.607232 8165 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.131
I20260812 02:36:00.617461 8165 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:43769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502156971023-7698-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502156971023-7698-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.131.keytab
--principal=kudu/127.7.132.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502156971023-7698-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.132.131
--webserver_port=0
--tserver_master_addrs=127.7.132.190:46247
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.131
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502156971023-7698-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:36:00.618922 8165 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:36:00.620329 8165 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:36:00.634282 8170 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:36:00.634208 8171 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:36:00.634248 8173 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:36:00.639241 8165 server_base.cc:1034] running on GCE node
I20260812 02:36:00.639868 8165 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:36:00.640869 8165 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:36:00.642167 8165 hybrid_clock.cc:648] HybridClock initialized: now 1786502160642031 us; error 49 us; skew 500 ppm
I20260812 02:36:00.642513 8165 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:36:00 dist-test-slave-mn4r krb5kdc[7706](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502160, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:36:00.645849 8165 init.cc:377] Logged in from keytab as kudu/127.7.132.131@KRBTEST.COM (short username kudu)
I20260812 02:36:00.647631 8165 webserver.cc:460] Webserver started at http://127.7.132.131:39089/ using document root <none> and password file <none>
I20260812 02:36:00.648415 8165 fs_manager.cc:359] Metadata directory not provided
I20260812 02:36:00.648591 8165 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:36:00.648931 8165 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:36:00.652063 8165 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502156971023-7698-0/minicluster-data/ts-2/data/instance:
uuid: "c3c31fc9016b497e82a13626924120ca"
format_stamp: "Formatted at 2026-08-12 02:36:00 on dist-test-slave-mn4r"
server_key: "f848f8cc5fabefdbd4977c271869b3f0"
server_key_iv: "2af49c4ee8160fde6b26468970736cfc"
server_key_version: "encryptionkey@0"
I20260812 02:36:00.652848 8165 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502156971023-7698-0/minicluster-data/ts-2/wal/instance:
uuid: "c3c31fc9016b497e82a13626924120ca"
format_stamp: "Formatted at 2026-08-12 02:36:00 on dist-test-slave-mn4r"
server_key: "f848f8cc5fabefdbd4977c271869b3f0"
server_key_iv: "2af49c4ee8160fde6b26468970736cfc"
server_key_version: "encryptionkey@0"
I20260812 02:36:00.658602 8165 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260812 02:36:00.662587 8181 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:36:00.664085 8165 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 02:36:00.664273 8165 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502156971023-7698-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502156971023-7698-0/minicluster-data/ts-2/wal
uuid: "c3c31fc9016b497e82a13626924120ca"
format_stamp: "Formatted at 2026-08-12 02:36:00 on dist-test-slave-mn4r"
server_key: "f848f8cc5fabefdbd4977c271869b3f0"
server_key_iv: "2af49c4ee8160fde6b26468970736cfc"
server_key_version: "encryptionkey@0"
I20260812 02:36:00.664466 8165 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502156971023-7698-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502156971023-7698-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502156971023-7698-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:36:00.670810 7989 heartbeater.cc:503] Master 127.7.132.190:46247 was elected leader, sending a full tablet report...
I20260812 02:36:00.676781 8165 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:36:00.681059 8165 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:36:00.681301 8165 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:36:00.682337 8165 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:36:00.683655 8165 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:36:00.683727 8165 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:36:00.683811 8165 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:36:00.683856 8165 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:36:00.719308 8165 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.131:40233
I20260812 02:36:00.719350 8320 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.131:40233 every 8 connection(s)
I20260812 02:36:00.720664 8165 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502156971023-7698-0/minicluster-data/ts-2/data/info.pb
I20260812 02:36:00.730551 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 8165
I20260812 02:36:00.730752 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786502156971023-7698-0/minicluster-data/ts-2/wal/instance
I20260812 02:36:00.731324 7698 external_mini_cluster.cc:1428] Setting key d262d2e67581c5f1febd560d324399da
Aug 12 02:36:00 dist-test-slave-mn4r krb5kdc[7706](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502160, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.131@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 02:36:00.742544 8327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:00.723433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:41091 (local address 127.7.132.190:46247)
0812 02:36:00.724865 (+ 1432us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:00.724873 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:00.725525 (+ 652us) server_negotiation.cc:407] Connection header received
0812 02:36:00.727557 (+ 2032us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:00.727563 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:00.727674 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:00.727802 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:00.729387 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:00.730302 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:00.732237 (+ 1935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:00.732794 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:00.732915 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:00.736871 (+ 3956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:00.736876 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:00.736879 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:00.736916 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:00.739628 (+ 2712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:00.740284 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:00.740288 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:00.740290 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:00.740431 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:00.740825 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:00.740828 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:00.740830 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:00.741101 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:00.741221 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:00.741922 (+ 701us) server_negotiation.cc:299] Negotiation successful
0812 02:36:00.742166 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1271,"thread_start_us":1098,"threads_started":1}
I20260812 02:36:00.743587 8328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:00.724200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:46247 (local address 127.7.132.131:41091)
0812 02:36:00.725270 (+ 1070us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:00.725336 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:00.727273 (+ 1937us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:00.728056 (+ 783us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:00.728065 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:00.728591 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:36:00.729190 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:00.729219 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:00.730470 (+ 1251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:00.730474 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:00.732033 (+ 1559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:00.732075 (+ 42us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:00.733041 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:00.733051 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:00.733291 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:00.734160 (+ 869us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:36:00.734189 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:36:00.736675 (+ 2486us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:36:00.739753 (+ 3078us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:00.739761 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:00.739772 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:00.740175 (+ 403us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:00.740534 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:00.740538 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:00.740540 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:00.740728 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:00.741215 (+ 487us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:36:00.741221 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:36:00.741544 (+ 323us) client_negotiation.cc:769] Sending connection context
0812 02:36:00.741795 (+ 251us) client_negotiation.cc:240] Negotiation successful
0812 02:36:00.742415 (+ 620us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":748,"thread_start_us":476,"threads_started":1}
I20260812 02:36:00.745044 8321 heartbeater.cc:347] Connected to a master server at 127.7.132.190:46247
I20260812 02:36:00.745359 8321 heartbeater.cc:464] Registering TS with master...
I20260812 02:36:00.746321 8321 heartbeater.cc:511] Master 127.7.132.190:46247 requested a full tablet report, sending...
I20260812 02:36:00.748152 7761 ts_manager.cc:194] Registered new tserver with Master: c3c31fc9016b497e82a13626924120ca (127.7.132.131:40233)
I20260812 02:36:00.749078 7761 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.131@KRBTEST.COM'} at 127.7.132.131:41091
I20260812 02:36:00.760257 7698 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 02:36:00 dist-test-slave-mn4r krb5kdc[7706](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502160, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 02:36:00 dist-test-slave-mn4r krb5kdc[7706](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502160, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.132.129@KRBTEST.COM
I20260812 02:36:00.810215 8334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:00.794449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46994 (local address 127.7.132.129:35513)
0812 02:36:00.795147 (+ 698us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:00.795155 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:00.795185 (+ 30us) server_negotiation.cc:407] Connection header received
0812 02:36:00.795342 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:00.795351 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:00.795580 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:00.795739 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:00.797257 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:00.798381 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:00.799650 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:00.800249 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:00.800348 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:00.804406 (+ 4058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:00.804417 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:00.804432 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:00.804472 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:00.807387 (+ 2915us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:00.808046 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:00.808053 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:00.808055 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:00.808197 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:00.808652 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:00.808655 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:00.808657 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:00.808990 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:00.809232 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:00.809514 (+ 282us) server_negotiation.cc:299] Negotiation successful
0812 02:36:00.809715 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":526,"thread_start_us":332,"threads_started":1}
W20260812 02:36:00.811658 7892 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46994
Aug 12 02:36:00 dist-test-slave-mn4r krb5kdc[7706](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502160, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 02:36:00 dist-test-slave-mn4r krb5kdc[7706](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502160, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.132.129@KRBTEST.COM
I20260812 02:36:00.859872 8334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:00.845704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47002 (local address 127.7.132.129:35513)
0812 02:36:00.845985 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:00.845993 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:00.846248 (+ 255us) server_negotiation.cc:407] Connection header received
0812 02:36:00.846508 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:00.846512 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:00.846613 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:00.846725 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:00.848150 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:00.848864 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:00.850120 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:00.850544 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:00.850678 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:00.854718 (+ 4040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:00.854724 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:00.854727 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:00.854761 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:00.857383 (+ 2622us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:00.858019 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:00.858023 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:00.858025 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:00.858305 (+ 280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:00.858637 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:00.858640 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:00.858642 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:00.858891 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:00.859072 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:00.859335 (+ 263us) server_negotiation.cc:299] Negotiation successful
0812 02:36:00.859515 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260812 02:36:00.861619 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 7837
I20260812 02:36:00.875034 8334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:00.862654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47012 (local address 127.7.132.129:35513)
0812 02:36:00.862879 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:00.862888 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:00.863217 (+ 329us) server_negotiation.cc:407] Connection header received
0812 02:36:00.863330 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:00.863333 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:00.863390 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:00.863487 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:00.864615 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:00.865310 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:00.866640 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:00.867091 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:00.867187 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:00.869819 (+ 2632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:00.869824 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:00.869827 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:00.869858 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:00.872509 (+ 2651us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:00.873365 (+ 856us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:00.873369 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:00.873371 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:00.873501 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:00.873826 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:00.873830 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:00.873831 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:00.874074 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:00.874313 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:00.874606 (+ 293us) server_negotiation.cc:299] Negotiation successful
0812 02:36:00.874768 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260812 02:36:00.875758 7984 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:36:00.960860 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 7837
I20260812 02:36:00.987676 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 7995
Aug 12 02:36:00 dist-test-slave-mn4r krb5kdc[7706](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502160, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.132.130@KRBTEST.COM
I20260812 02:36:01.004421 8341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:00.988582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37014 (local address 127.7.132.130:44211)
0812 02:36:00.989164 (+ 582us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:00.989172 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:00.989198 (+ 26us) server_negotiation.cc:407] Connection header received
0812 02:36:00.989262 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:00.989268 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:00.989486 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:00.989633 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:00.991145 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:00.992171 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:00.993428 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:00.993998 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:00.994210 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:00.998287 (+ 4077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:00.998300 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:00.998315 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:00.998359 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:01.001384 (+ 3025us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:01.002265 (+ 881us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:01.002275 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:01.002277 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:01.002426 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:01.002922 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:01.002925 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:01.002927 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:01.003343 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:01.003600 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:01.003869 (+ 269us) server_negotiation.cc:299] Negotiation successful
0812 02:36:01.004104 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":408,"thread_start_us":301,"threads_started":1}
I20260812 02:36:01.005815 8146 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:36:01.089628 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 7995
I20260812 02:36:01.110877 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 8165
Aug 12 02:36:01 dist-test-slave-mn4r krb5kdc[7706](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502160, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.132.131@KRBTEST.COM
I20260812 02:36:01.128681 8343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:01.112102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48606 (local address 127.7.132.131:40233)
0812 02:36:01.112794 (+ 692us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:01.112801 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:01.112832 (+ 31us) server_negotiation.cc:407] Connection header received
0812 02:36:01.112898 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:01.112904 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:01.113126 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:01.113322 (+ 196us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:01.114817 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:01.115888 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:01.117249 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:01.117863 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:01.118031 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:01.122705 (+ 4674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:01.122719 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:01.122735 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:01.122773 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:01.125753 (+ 2980us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:01.126561 (+ 808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:01.126569 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:01.126571 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:01.126729 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:01.127111 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:01.127114 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:01.127116 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:01.127536 (+ 420us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:01.127862 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:01.128057 (+ 195us) server_negotiation.cc:299] Negotiation successful
0812 02:36:01.128270 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":537,"thread_start_us":369,"threads_started":1}
I20260812 02:36:01.130168 8316 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:36:01.214521 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 8165
I20260812 02:36:01.235455 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 7722
I20260812 02:36:01.236778 7796 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:36:01.342741 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 7722
2026-08-12T02:36:01Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (4345 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502156971023-7698-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 02:36:01 dist-test-slave-mn4r krb5kdc[8350](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 02:36:01 dist-test-slave-mn4r krb5kdc[8350](info): set up 2 sockets
Aug 12 02:36:01 dist-test-slave-mn4r krb5kdc[8350](info): commencing operation
krb5kdc: starting...
W20260812 02:36:02.536810 7698 mini_kdc.cc:121] Time spent starting KDC: real 1.142s user 0.002s sys 0.018s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502156971023-7698-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502156971023-7698-0/krb5kdc/test-user.keytab.
Aug 12 02:36:02 dist-test-slave-mn4r krb5kdc[8350](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502162, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T02:36:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T02:36:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.132.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
I20260812 02:36:02.748245 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502156971023-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502156971023-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502156971023-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:34915
--webserver_interface=127.7.132.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:44327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.132.190:34915
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:36:03.088202 8364 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:36:03.088518 8364 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:36:03.088688 8364 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:36:03.096391 8364 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:36:03.096517 8364 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:36:03.096614 8364 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:36:03.096724 8364 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:36:03.096802 8364 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:36:03.107316 8364 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:44327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502156971023-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502156971023-7698-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.132.190:34915
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:34915
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.132.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502156971023-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:36:03.108754 8364 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:36:03.110275 8364 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:36:03.122705 8372 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:36:03.122705 8375 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:36:03.123039 8373 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:36:03.124022 8364 server_base.cc:1034] running on GCE node
I20260812 02:36:03.124856 8364 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:36:03.126385 8364 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:36:03.127631 8364 hybrid_clock.cc:648] HybridClock initialized: now 1786502163127584 us; error 36 us; skew 500 ppm
I20260812 02:36:03.128002 8364 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:36:03 dist-test-slave-mn4r krb5kdc[8350](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502163, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:36:03.136471 8364 init.cc:377] Logged in from keytab as kudu/127.7.132.190@KRBTEST.COM (short username kudu)
I20260812 02:36:03.138361 8364 webserver.cc:460] Webserver started at http://127.7.132.190:40459/ using document root <none> and password file <none>
I20260812 02:36:03.139122 8364 fs_manager.cc:359] Metadata directory not provided
I20260812 02:36:03.139276 8364 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:36:03.139611 8364 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:36:03.142882 8364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502156971023-7698-0/minicluster-data/master-0/data/instance:
uuid: "8b043ff606c84d2aaf922b6036bdc458"
format_stamp: "Formatted at 2026-08-12 02:36:03 on dist-test-slave-mn4r"
server_key: "a0dfe6a208cc431322c8e1a2fc48039e"
server_key_iv: "b91419c7e3abd2cf1007178171f8413f"
server_key_version: "encryptionkey@0"
I20260812 02:36:03.143664 8364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502156971023-7698-0/minicluster-data/master-0/wal/instance:
uuid: "8b043ff606c84d2aaf922b6036bdc458"
format_stamp: "Formatted at 2026-08-12 02:36:03 on dist-test-slave-mn4r"
server_key: "a0dfe6a208cc431322c8e1a2fc48039e"
server_key_iv: "b91419c7e3abd2cf1007178171f8413f"
server_key_version: "encryptionkey@0"
I20260812 02:36:03.149019 8364 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260812 02:36:03.152866 8382 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:36:03.154402 8364 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260812 02:36:03.154584 8364 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502156971023-7698-0/minicluster-data/master-0/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502156971023-7698-0/minicluster-data/master-0/wal
uuid: "8b043ff606c84d2aaf922b6036bdc458"
format_stamp: "Formatted at 2026-08-12 02:36:03 on dist-test-slave-mn4r"
server_key: "a0dfe6a208cc431322c8e1a2fc48039e"
server_key_iv: "b91419c7e3abd2cf1007178171f8413f"
server_key_version: "encryptionkey@0"
I20260812 02:36:03.154803 8364 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502156971023-7698-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502156971023-7698-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502156971023-7698-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:36:03.197726 8364 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:36:03.202546 8364 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:36:03.202860 8364 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:36:03.221553 8364 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.190:34915
I20260812 02:36:03.221565 8449 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.190:34915 every 8 connection(s)
I20260812 02:36:03.223069 8364 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
I20260812 02:36:03.227221 8451 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:36:03.230639 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 8364
I20260812 02:36:03.230890 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502156971023-7698-0/minicluster-data/master-0/wal/instance
I20260812 02:36:03.231559 7698 external_mini_cluster.cc:1428] Setting key 8af5cc8822e6693908e2cb88d66229b4
I20260812 02:36:03.240675 8451 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b043ff606c84d2aaf922b6036bdc458: Bootstrap starting.
I20260812 02:36:03.244807 8451 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8b043ff606c84d2aaf922b6036bdc458: Neither blocks nor log segments found. Creating new log.
I20260812 02:36:03.246272 8451 log.cc:824] T 00000000000000000000000000000000 P 8b043ff606c84d2aaf922b6036bdc458: Log is configured to *not* fsync() on all Append() calls
Aug 12 02:36:03 dist-test-slave-mn4r krb5kdc[8350](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502162, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 02:36:03.249418 8451 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b043ff606c84d2aaf922b6036bdc458: No bootstrap required, opened a new log
I20260812 02:36:03.255893 8454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:03.233661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40524 (local address 127.7.132.190:34915)
0812 02:36:03.234650 (+ 989us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:03.234663 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:03.234706 (+ 43us) server_negotiation.cc:407] Connection header received
0812 02:36:03.236349 (+ 1643us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:03.236373 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:03.236810 (+ 437us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:03.237157 (+ 347us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:03.239060 (+ 1903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:03.240403 (+ 1343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:03.242247 (+ 1844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:03.242997 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:03.243141 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:03.248192 (+ 5051us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:03.248201 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:03.248218 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:03.248255 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:03.251515 (+ 3260us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:03.252301 (+ 786us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:03.252309 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:03.252315 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:03.252496 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:03.252903 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:03.252915 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:03.252918 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:03.253465 (+ 547us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:03.253833 (+ 368us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:03.254366 (+ 533us) server_negotiation.cc:299] Negotiation successful
0812 02:36:03.254804 (+ 438us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":742,"thread_start_us":541,"threads_started":1}
I20260812 02:36:03.257910 8451 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8b043ff606c84d2aaf922b6036bdc458 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b043ff606c84d2aaf922b6036bdc458" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 34915 } }
I20260812 02:36:03.258385 8451 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8b043ff606c84d2aaf922b6036bdc458 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:36:03.258522 8451 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8b043ff606c84d2aaf922b6036bdc458 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b043ff606c84d2aaf922b6036bdc458, State: Initialized, Role: FOLLOWER
I20260812 02:36:03.259274 8451 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8b043ff606c84d2aaf922b6036bdc458 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b043ff606c84d2aaf922b6036bdc458" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 34915 } }
I20260812 02:36:03.259577 8451 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8b043ff606c84d2aaf922b6036bdc458 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 02:36:03.259722 8451 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8b043ff606c84d2aaf922b6036bdc458 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 02:36:03.259887 8451 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8b043ff606c84d2aaf922b6036bdc458 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:36:03.262250 8451 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8b043ff606c84d2aaf922b6036bdc458 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b043ff606c84d2aaf922b6036bdc458" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 34915 } }
I20260812 02:36:03.262811 8451 leader_election.cc:302] T 00000000000000000000000000000000 P 8b043ff606c84d2aaf922b6036bdc458 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8b043ff606c84d2aaf922b6036bdc458; no voters:
I20260812 02:36:03.263564 8451 leader_election.cc:288] T 00000000000000000000000000000000 P 8b043ff606c84d2aaf922b6036bdc458 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 02:36:03.263695 8457 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8b043ff606c84d2aaf922b6036bdc458 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:36:03.264344 8457 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8b043ff606c84d2aaf922b6036bdc458 [term 1 LEADER]: Becoming Leader. State: Replica: 8b043ff606c84d2aaf922b6036bdc458, State: Running, Role: LEADER
I20260812 02:36:03.264892 8457 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8b043ff606c84d2aaf922b6036bdc458 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b043ff606c84d2aaf922b6036bdc458" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 34915 } }
I20260812 02:36:03.265866 8451 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8b043ff606c84d2aaf922b6036bdc458 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:36:03.267917 8458 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b043ff606c84d2aaf922b6036bdc458 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8b043ff606c84d2aaf922b6036bdc458" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b043ff606c84d2aaf922b6036bdc458" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 34915 } } }
I20260812 02:36:03.268002 8459 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b043ff606c84d2aaf922b6036bdc458 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8b043ff606c84d2aaf922b6036bdc458. Latest consensus state: current_term: 1 leader_uuid: "8b043ff606c84d2aaf922b6036bdc458" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b043ff606c84d2aaf922b6036bdc458" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 34915 } } }
I20260812 02:36:03.268411 8459 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b043ff606c84d2aaf922b6036bdc458 [sys.catalog]: This master's current role is: LEADER
I20260812 02:36:03.268289 8458 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b043ff606c84d2aaf922b6036bdc458 [sys.catalog]: This master's current role is: LEADER
I20260812 02:36:03.271756 8464 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:36:03.276772 8464 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 02:36:03.286149 8464 catalog_manager.cc:1347] Generated new cluster ID: e55ae771f9a64f09a19600cd00ce4483
I20260812 02:36:03.286273 8464 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 02:36:03.294255 8464 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 02:36:03.295655 8464 catalog_manager.cc:1504] Loading token signing keys...
I20260812 02:36:03.302583 8464 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8b043ff606c84d2aaf922b6036bdc458: Generated new TSK 0
I20260812 02:36:03.303474 8464 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.132.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.129.keytab.
Entry for principal kudu/127.7.132.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.129.keytab.
I20260812 02:36:03.370549 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502156971023-7698-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502156971023-7698-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502156971023-7698-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502156971023-7698-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.129:0
--local_ip_for_outbound_sockets=127.7.132.129
--webserver_interface=127.7.132.129
--webserver_port=0
--tserver_master_addrs=127.7.132.190:34915
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.129.keytab
--principal=kudu/127.7.132.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:44327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:36:03.713750 8484 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:36:03.714133 8484 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:36:03.714318 8484 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:36:03.721923 8484 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:36:03.722146 8484 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:36:03.722333 8484 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.129
I20260812 02:36:03.732668 8484 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:44327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502156971023-7698-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502156971023-7698-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.129.keytab
--principal=kudu/127.7.132.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502156971023-7698-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.132.129
--webserver_port=0
--tserver_master_addrs=127.7.132.190:34915
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.129
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502156971023-7698-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:36:03.734153 8484 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:36:03.735569 8484 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:36:03.749622 8490 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:36:03.749542 8491 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:36:03.749616 8494 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:36:03.750475 8484 server_base.cc:1034] running on GCE node
I20260812 02:36:03.751139 8484 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:36:03.752207 8484 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:36:03.753458 8484 hybrid_clock.cc:648] HybridClock initialized: now 1786502163753408 us; error 43 us; skew 500 ppm
I20260812 02:36:03.753854 8484 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:36:03 dist-test-slave-mn4r krb5kdc[8350](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502163, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:36:03.757694 8484 init.cc:377] Logged in from keytab as kudu/127.7.132.129@KRBTEST.COM (short username kudu)
I20260812 02:36:03.759547 8484 webserver.cc:460] Webserver started at http://127.7.132.129:34045/ using document root <none> and password file <none>
I20260812 02:36:03.760324 8484 fs_manager.cc:359] Metadata directory not provided
I20260812 02:36:03.760478 8484 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:36:03.760828 8484 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:36:03.764065 8484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502156971023-7698-0/minicluster-data/ts-0/data/instance:
uuid: "f80db53180114a2f9b110d710696a857"
format_stamp: "Formatted at 2026-08-12 02:36:03 on dist-test-slave-mn4r"
server_key: "85c0bf5359379954bf752eee8a23cdcf"
server_key_iv: "d340be1fc2608a29cd36ca5bfe9a9f88"
server_key_version: "encryptionkey@0"
I20260812 02:36:03.764925 8484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502156971023-7698-0/minicluster-data/ts-0/wal/instance:
uuid: "f80db53180114a2f9b110d710696a857"
format_stamp: "Formatted at 2026-08-12 02:36:03 on dist-test-slave-mn4r"
server_key: "85c0bf5359379954bf752eee8a23cdcf"
server_key_iv: "d340be1fc2608a29cd36ca5bfe9a9f88"
server_key_version: "encryptionkey@0"
I20260812 02:36:03.771046 8484 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260812 02:36:03.774912 8503 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:36:03.776439 8484 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 02:36:03.776631 8484 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502156971023-7698-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502156971023-7698-0/minicluster-data/ts-0/wal
uuid: "f80db53180114a2f9b110d710696a857"
format_stamp: "Formatted at 2026-08-12 02:36:03 on dist-test-slave-mn4r"
server_key: "85c0bf5359379954bf752eee8a23cdcf"
server_key_iv: "d340be1fc2608a29cd36ca5bfe9a9f88"
server_key_version: "encryptionkey@0"
I20260812 02:36:03.776846 8484 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502156971023-7698-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502156971023-7698-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502156971023-7698-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:36:03.797833 8484 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:36:03.802938 8484 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:36:03.803256 8484 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:36:03.804304 8484 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:36:03.805791 8484 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:36:03.805867 8484 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:36:03.806049 8484 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:36:03.806157 8484 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260812 02:36:03.840442 8484 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.129:40903
I20260812 02:36:03.840489 8643 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.129:40903 every 8 connection(s)
I20260812 02:36:03.841907 8484 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502156971023-7698-0/minicluster-data/ts-0/data/info.pb
I20260812 02:36:03.842805 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 8484
I20260812 02:36:03.843065 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502156971023-7698-0/minicluster-data/ts-0/wal/instance
I20260812 02:36:03.843781 7698 external_mini_cluster.cc:1428] Setting key afea9579731db37e955f04c4a009e7e5
Aug 12 02:36:03 dist-test-slave-mn4r krb5kdc[8350](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502163, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.129@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.132.130@KRBTEST.COM; defaulting to no policy
I20260812 02:36:03.865175 8647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:03.844931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:43333 (local address 127.7.132.190:34915)
0812 02:36:03.845449 (+ 518us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:03.845455 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:03.846669 (+ 1214us) server_negotiation.cc:407] Connection header received
0812 02:36:03.848572 (+ 1903us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:03.848582 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:03.848688 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:03.848816 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:03.850465 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:03.851659 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:03.853266 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:03.854027 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:03.854206 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:03.857797 (+ 3591us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:03.857806 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:03.857814 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:03.857862 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:03.861727 (+ 3865us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:03.862583 (+ 856us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:03.862591 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:03.862597 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:03.862765 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:03.863214 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:03.863220 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:03.863224 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:03.863629 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:03.863799 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:03.864559 (+ 760us) server_negotiation.cc:299] Negotiation successful
0812 02:36:03.864807 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":256,"threads_started":1}
I20260812 02:36:03.866430 8648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:03.845571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:34915 (local address 127.7.132.129:43333)
0812 02:36:03.846466 (+ 895us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:03.846525 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:03.848271 (+ 1746us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:03.849096 (+ 825us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:03.849105 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:03.849628 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:36:03.850285 (+ 657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:03.850306 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:03.851839 (+ 1533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:03.851843 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:03.853106 (+ 1263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:03.853118 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:03.854304 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:03.854310 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:03.854536 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:03.855272 (+ 736us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:36:03.855301 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:36:03.857543 (+ 2242us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:36:03.861916 (+ 4373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:03.861923 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:03.861937 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:03.862445 (+ 508us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:03.862890 (+ 445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:03.862893 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:03.862896 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:03.863082 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:03.863772 (+ 690us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:36:03.863778 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:36:03.864143 (+ 365us) client_negotiation.cc:769] Sending connection context
0812 02:36:03.864428 (+ 285us) client_negotiation.cc:240] Negotiation successful
0812 02:36:03.864996 (+ 568us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":615,"thread_start_us":350,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.130@KRBTEST.COM" created.
I20260812 02:36:03.868369 8644 heartbeater.cc:347] Connected to a master server at 127.7.132.190:34915
I20260812 02:36:03.868714 8644 heartbeater.cc:464] Registering TS with master...
I20260812 02:36:03.869531 8644 heartbeater.cc:511] Master 127.7.132.190:34915 requested a full tablet report, sending...
I20260812 02:36:03.872035 8408 ts_manager.cc:194] Registered new tserver with Master: f80db53180114a2f9b110d710696a857 (127.7.132.129:40903)
I20260812 02:36:03.873980 8408 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.129@KRBTEST.COM'} at 127.7.132.129:43333
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.130.keytab.
Entry for principal kudu/127.7.132.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.130.keytab.
I20260812 02:36:03.898154 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502156971023-7698-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502156971023-7698-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502156971023-7698-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502156971023-7698-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.130:0
--local_ip_for_outbound_sockets=127.7.132.130
--webserver_interface=127.7.132.130
--webserver_port=0
--tserver_master_addrs=127.7.132.190:34915
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.130.keytab
--principal=kudu/127.7.132.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:44327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:36:04.233889 8651 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:36:04.234277 8651 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:36:04.234462 8651 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:36:04.241801 8651 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:36:04.242018 8651 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:36:04.242267 8651 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.130
I20260812 02:36:04.252115 8651 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:44327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502156971023-7698-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502156971023-7698-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.130.keytab
--principal=kudu/127.7.132.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502156971023-7698-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.132.130
--webserver_port=0
--tserver_master_addrs=127.7.132.190:34915
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.130
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502156971023-7698-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:36:04.253502 8651 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:36:04.254931 8651 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:36:04.269155 8657 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:36:04.269106 8660 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:36:04.269198 8658 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:36:04.270004 8651 server_base.cc:1034] running on GCE node
I20260812 02:36:04.270771 8651 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:36:04.271700 8651 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:36:04.272941 8651 hybrid_clock.cc:648] HybridClock initialized: now 1786502164272868 us; error 83 us; skew 500 ppm
I20260812 02:36:04.273277 8651 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:36:04 dist-test-slave-mn4r krb5kdc[8350](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502164, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:36:04.276582 8651 init.cc:377] Logged in from keytab as kudu/127.7.132.130@KRBTEST.COM (short username kudu)
I20260812 02:36:04.278162 8651 webserver.cc:460] Webserver started at http://127.7.132.130:34921/ using document root <none> and password file <none>
I20260812 02:36:04.278854 8651 fs_manager.cc:359] Metadata directory not provided
I20260812 02:36:04.278959 8651 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:36:04.279223 8651 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:36:04.282265 8651 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502156971023-7698-0/minicluster-data/ts-1/data/instance:
uuid: "2ae594811b024a34aff5ea4109c4c586"
format_stamp: "Formatted at 2026-08-12 02:36:04 on dist-test-slave-mn4r"
server_key: "30e9d1653f8cfa7c6111daf74f6923b0"
server_key_iv: "314d9a159a2f70796cfc378ee20fa9f8"
server_key_version: "encryptionkey@0"
I20260812 02:36:04.283004 8651 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502156971023-7698-0/minicluster-data/ts-1/wal/instance:
uuid: "2ae594811b024a34aff5ea4109c4c586"
format_stamp: "Formatted at 2026-08-12 02:36:04 on dist-test-slave-mn4r"
server_key: "30e9d1653f8cfa7c6111daf74f6923b0"
server_key_iv: "314d9a159a2f70796cfc378ee20fa9f8"
server_key_version: "encryptionkey@0"
I20260812 02:36:04.288151 8651 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260812 02:36:04.291435 8667 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:36:04.292781 8651 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 02:36:04.292909 8651 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502156971023-7698-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502156971023-7698-0/minicluster-data/ts-1/wal
uuid: "2ae594811b024a34aff5ea4109c4c586"
format_stamp: "Formatted at 2026-08-12 02:36:04 on dist-test-slave-mn4r"
server_key: "30e9d1653f8cfa7c6111daf74f6923b0"
server_key_iv: "314d9a159a2f70796cfc378ee20fa9f8"
server_key_version: "encryptionkey@0"
I20260812 02:36:04.293072 8651 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502156971023-7698-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502156971023-7698-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502156971023-7698-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:36:04.316995 8651 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:36:04.321170 8651 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:36:04.321419 8651 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:36:04.322549 8651 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:36:04.323908 8651 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:36:04.323985 8651 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:36:04.324069 8651 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:36:04.324115 8651 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:36:04.358433 8651 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.130:43411
I20260812 02:36:04.358578 8810 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.130:43411 every 8 connection(s)
I20260812 02:36:04.359907 8651 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502156971023-7698-0/minicluster-data/ts-1/data/info.pb
I20260812 02:36:04.367837 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 8651
I20260812 02:36:04.368059 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502156971023-7698-0/minicluster-data/ts-1/wal/instance
I20260812 02:36:04.368520 7698 external_mini_cluster.cc:1428] Setting key 1ac3fb4f15a6d0564b3bf0dd6543099a
Aug 12 02:36:04 dist-test-slave-mn4r krb5kdc[8350](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502164, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.130@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 02:36:04.382913 8647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:04.363138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:53207 (local address 127.7.132.190:34915)
0812 02:36:04.363437 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:04.363443 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:04.364722 (+ 1279us) server_negotiation.cc:407] Connection header received
0812 02:36:04.366728 (+ 2006us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:04.366734 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:04.366826 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:04.366957 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:04.368589 (+ 1632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:04.369526 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:04.371376 (+ 1850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:04.371817 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:04.371962 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:04.375690 (+ 3728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:04.375695 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:04.375698 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:04.375731 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:04.379299 (+ 3568us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:04.380042 (+ 743us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:04.380050 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:04.380057 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:04.380217 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:04.380697 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:04.380704 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:04.380711 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:04.381147 (+ 436us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:04.381307 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:04.382151 (+ 844us) server_negotiation.cc:299] Negotiation successful
0812 02:36:04.382452 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260812 02:36:04.383551 8815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:04.363811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:34915 (local address 127.7.132.130:53207)
0812 02:36:04.364541 (+ 730us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:04.364584 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:04.366432 (+ 1848us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:04.367199 (+ 767us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:04.367208 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:04.367728 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:36:04.368408 (+ 680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:04.368429 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:04.369685 (+ 1256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:04.369692 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:04.371223 (+ 1531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:04.371245 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:04.372056 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:04.372064 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:04.372331 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:04.373056 (+ 725us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:36:04.373079 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:36:04.375441 (+ 2362us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:36:04.379439 (+ 3998us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:04.379445 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:04.379458 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:04.379900 (+ 442us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:04.380364 (+ 464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:04.380368 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:04.380370 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:04.380573 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:04.381287 (+ 714us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:36:04.381292 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:36:04.381669 (+ 377us) client_negotiation.cc:769] Sending connection context
0812 02:36:04.381962 (+ 293us) client_negotiation.cc:240] Negotiation successful
0812 02:36:04.382412 (+ 450us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":269,"threads_started":1}
I20260812 02:36:04.385128 8812 heartbeater.cc:347] Connected to a master server at 127.7.132.190:34915
I20260812 02:36:04.385478 8812 heartbeater.cc:464] Registering TS with master...
I20260812 02:36:04.386474 8812 heartbeater.cc:511] Master 127.7.132.190:34915 requested a full tablet report, sending...
I20260812 02:36:04.388092 8408 ts_manager.cc:194] Registered new tserver with Master: 2ae594811b024a34aff5ea4109c4c586 (127.7.132.130:43411)
I20260812 02:36:04.388919 8408 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.130@KRBTEST.COM'} at 127.7.132.130:53207
WARNING: no policy specified for kudu/127.7.132.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.131.keytab.
Entry for principal kudu/127.7.132.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.131.keytab.
I20260812 02:36:04.417321 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502156971023-7698-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502156971023-7698-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502156971023-7698-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502156971023-7698-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.131:0
--local_ip_for_outbound_sockets=127.7.132.131
--webserver_interface=127.7.132.131
--webserver_port=0
--tserver_master_addrs=127.7.132.190:34915
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.131.keytab
--principal=kudu/127.7.132.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:44327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:36:04.759232 8819 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:36:04.759548 8819 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:36:04.759755 8819 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:36:04.767043 8819 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:36:04.767195 8819 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:36:04.767365 8819 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.131
I20260812 02:36:04.777490 8819 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:44327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502156971023-7698-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502156971023-7698-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.131.keytab
--principal=kudu/127.7.132.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502156971023-7698-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.132.131
--webserver_port=0
--tserver_master_addrs=127.7.132.190:34915
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.131
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502156971023-7698-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:36:04.778954 8819 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:36:04.780257 8819 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:36:04.794106 8828 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:36:04.794019 8825 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:36:04.794139 8826 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:36:04.794899 8819 server_base.cc:1034] running on GCE node
I20260812 02:36:04.795565 8819 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:36:04.796663 8819 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:36:04.797883 8819 hybrid_clock.cc:648] HybridClock initialized: now 1786502164797793 us; error 79 us; skew 500 ppm
I20260812 02:36:04.798265 8819 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:36:04 dist-test-slave-mn4r krb5kdc[8350](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502164, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:36:04.801925 8819 init.cc:377] Logged in from keytab as kudu/127.7.132.131@KRBTEST.COM (short username kudu)
I20260812 02:36:04.803521 8819 webserver.cc:460] Webserver started at http://127.7.132.131:33459/ using document root <none> and password file <none>
I20260812 02:36:04.804225 8819 fs_manager.cc:359] Metadata directory not provided
I20260812 02:36:04.804335 8819 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:36:04.804622 8819 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:36:04.807916 8819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502156971023-7698-0/minicluster-data/ts-2/data/instance:
uuid: "4579e6b2dea1461dbd4c22e57ee97f2d"
format_stamp: "Formatted at 2026-08-12 02:36:04 on dist-test-slave-mn4r"
server_key: "2535ed20ab2cc0a75f8ed6d961486eef"
server_key_iv: "d3e1e6beef4b794bacb37e4c6d42c6b9"
server_key_version: "encryptionkey@0"
I20260812 02:36:04.808738 8819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502156971023-7698-0/minicluster-data/ts-2/wal/instance:
uuid: "4579e6b2dea1461dbd4c22e57ee97f2d"
format_stamp: "Formatted at 2026-08-12 02:36:04 on dist-test-slave-mn4r"
server_key: "2535ed20ab2cc0a75f8ed6d961486eef"
server_key_iv: "d3e1e6beef4b794bacb37e4c6d42c6b9"
server_key_version: "encryptionkey@0"
I20260812 02:36:04.814309 8819 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260812 02:36:04.817974 8835 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:36:04.819378 8819 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 02:36:04.819574 8819 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502156971023-7698-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502156971023-7698-0/minicluster-data/ts-2/wal
uuid: "4579e6b2dea1461dbd4c22e57ee97f2d"
format_stamp: "Formatted at 2026-08-12 02:36:04 on dist-test-slave-mn4r"
server_key: "2535ed20ab2cc0a75f8ed6d961486eef"
server_key_iv: "d3e1e6beef4b794bacb37e4c6d42c6b9"
server_key_version: "encryptionkey@0"
I20260812 02:36:04.819810 8819 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502156971023-7698-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502156971023-7698-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502156971023-7698-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:36:04.839550 8819 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:36:04.845480 8819 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:36:04.845819 8819 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:36:04.847128 8819 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:36:04.848739 8819 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:36:04.848821 8819 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:36:04.848924 8819 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:36:04.849015 8819 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:36:04.877909 8644 heartbeater.cc:503] Master 127.7.132.190:34915 was elected leader, sending a full tablet report...
I20260812 02:36:04.890853 8819 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.131:37211
I20260812 02:36:04.890899 8973 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.131:37211 every 8 connection(s)
I20260812 02:36:04.892355 8819 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502156971023-7698-0/minicluster-data/ts-2/data/info.pb
I20260812 02:36:04.897817 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 8819
I20260812 02:36:04.898034 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786502156971023-7698-0/minicluster-data/ts-2/wal/instance
I20260812 02:36:04.898540 7698 external_mini_cluster.cc:1428] Setting key 0f1fc70a8106ea8d75a4fcf34b6244c5
Aug 12 02:36:04 dist-test-slave-mn4r krb5kdc[8350](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502164, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.131@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 02:36:04.914940 8977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:04.895330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:60713 (local address 127.7.132.190:34915)
0812 02:36:04.895905 (+ 575us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:04.895913 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:04.897134 (+ 1221us) server_negotiation.cc:407] Connection header received
0812 02:36:04.899337 (+ 2203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:04.899345 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:04.899450 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:04.899588 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:04.901750 (+ 2162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:04.902783 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:04.904352 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:04.904795 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:04.904899 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:04.908454 (+ 3555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:04.908459 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:04.908463 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:04.908501 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:04.911266 (+ 2765us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:04.911958 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:04.911962 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:04.911965 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:04.912143 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:04.912787 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:04.912792 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:04.912796 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:04.913221 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:04.913374 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:04.914342 (+ 968us) server_negotiation.cc:299] Negotiation successful
0812 02:36:04.914568 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":436,"thread_start_us":301,"threads_started":1}
I20260812 02:36:04.915932 8978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:04.896008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:34915 (local address 127.7.132.131:60713)
0812 02:36:04.896937 (+ 929us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:04.896987 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:04.898966 (+ 1979us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:04.899908 (+ 942us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:04.899916 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:04.900673 (+ 757us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:36:04.901548 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:04.901578 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:04.902936 (+ 1358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:04.902941 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:04.904215 (+ 1274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:04.904228 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:04.904995 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:04.905000 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:04.905206 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:04.905914 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:36:04.905937 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:36:04.908231 (+ 2294us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:36:04.911399 (+ 3168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:04.911406 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:04.911418 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:04.911840 (+ 422us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:04.912261 (+ 421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:04.912265 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:04.912267 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:04.912651 (+ 384us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:04.913368 (+ 717us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:36:04.913374 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:36:04.913817 (+ 443us) client_negotiation.cc:769] Sending connection context
0812 02:36:04.914207 (+ 390us) client_negotiation.cc:240] Negotiation successful
0812 02:36:04.914754 (+ 547us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":602,"thread_start_us":402,"threads_started":1}
I20260812 02:36:04.917353 8974 heartbeater.cc:347] Connected to a master server at 127.7.132.190:34915
I20260812 02:36:04.917691 8974 heartbeater.cc:464] Registering TS with master...
I20260812 02:36:04.918519 8974 heartbeater.cc:511] Master 127.7.132.190:34915 requested a full tablet report, sending...
I20260812 02:36:04.919950 8408 ts_manager.cc:194] Registered new tserver with Master: 4579e6b2dea1461dbd4c22e57ee97f2d (127.7.132.131:37211)
I20260812 02:36:04.920771 8408 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.131@KRBTEST.COM'} at 127.7.132.131:60713
I20260812 02:36:04.927153 7698 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 02:36:04.947666 8977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:04.933877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40536 (local address 127.7.132.190:34915)
0812 02:36:04.934053 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:04.934058 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:04.934604 (+ 546us) server_negotiation.cc:407] Connection header received
0812 02:36:04.934819 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:04.934828 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:04.934900 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:04.935014 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:04.936636 (+ 1622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:04.937437 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:04.938934 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:04.939400 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:04.939564 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:04.942216 (+ 2652us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:04.942222 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:04.942225 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:04.942259 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:04.944849 (+ 2590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:04.945536 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:04.945542 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:04.945546 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:04.945675 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:04.946097 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:04.946105 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:04.946109 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:04.946438 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:04.946714 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:04.947076 (+ 362us) server_negotiation.cc:299] Negotiation successful
0812 02:36:04.947323 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260812 02:36:04.954782 8408 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40536:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 02:36:04.962765 8408 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 02:36:04.999468 8990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:04.976006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.130:43411 (local address 127.0.0.1:60692)
0812 02:36:04.986757 (+ 10751us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:04.986792 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:04.987116 (+ 324us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:04.989848 (+ 2732us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:04.989863 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:04.989910 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:36:04.990515 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:04.990534 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:04.993382 (+ 2848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:04.993388 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:04.996535 (+ 3147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:04.996554 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:04.998535 (+ 1981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:04.998543 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:04.998750 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:04.998806 (+ 56us) client_negotiation.cc:769] Sending connection context
0812 02:36:04.998965 (+ 159us) client_negotiation.cc:240] Negotiation successful
0812 02:36:04.999135 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10556,"mutex_wait_us":1,"thread_start_us":6829,"threads_started":1}
I20260812 02:36:05.000293 8991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:04.975970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60692 (local address 127.7.132.130:43411)
0812 02:36:04.976572 (+ 602us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:04.976583 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:04.986961 (+ 10378us) server_negotiation.cc:407] Connection header received
0812 02:36:04.988764 (+ 1803us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:04.988779 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:04.989163 (+ 384us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:04.989397 (+ 234us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:36:04.990773 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:04.993124 (+ 2351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:04.996700 (+ 3576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:04.998306 (+ 1606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:04.998492 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:04.998714 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:04.999762 (+ 1048us) server_negotiation.cc:299] Negotiation successful
0812 02:36:04.999937 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":393,"thread_start_us":279,"threads_started":1}
I20260812 02:36:05.002643 8993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:04.975874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:40903 (local address 127.0.0.1:43398)
0812 02:36:04.983339 (+ 7465us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:04.983407 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:04.983665 (+ 258us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:04.994908 (+ 11243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:04.994913 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:04.994940 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:36:04.995495 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:04.995508 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:04.997510 (+ 2002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:04.997519 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:04.999886 (+ 2367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:04.999904 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:05.001739 (+ 1835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:05.001747 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:05.001946 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:05.001991 (+ 45us) client_negotiation.cc:769] Sending connection context
0812 02:36:05.002127 (+ 136us) client_negotiation.cc:240] Negotiation successful
0812 02:36:05.002256 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7126,"mutex_wait_us":83,"thread_start_us":10472,"threads_started":1}
I20260812 02:36:05.002658 8997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:04.977041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43398 (local address 127.7.132.129:40903)
0812 02:36:04.994267 (+ 17226us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:04.994276 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:04.994306 (+ 30us) server_negotiation.cc:407] Connection header received
0812 02:36:04.994405 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:04.994414 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:04.994688 (+ 274us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:04.994926 (+ 238us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:36:04.995753 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:04.997364 (+ 1611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:05.000062 (+ 2698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:05.001522 (+ 1460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:05.001674 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:05.001928 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:05.002166 (+ 238us) server_negotiation.cc:299] Negotiation successful
0812 02:36:05.002326 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16953,"thread_start_us":16289,"threads_started":1}
I20260812 02:36:05.004016 8995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:04.976456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.131:37211 (local address 127.0.0.1:53088)
0812 02:36:04.994185 (+ 17729us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:04.994209 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:04.994342 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:04.995191 (+ 849us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:04.995195 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:04.995241 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:36:04.995742 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:04.995757 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:04.997575 (+ 1818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:04.997586 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:05.001262 (+ 3676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:05.001274 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:05.003354 (+ 2080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:05.003359 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:05.003494 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:05.003517 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 02:36:05.003578 (+ 61us) client_negotiation.cc:240] Negotiation successful
0812 02:36:05.003730 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17555,"mutex_wait_us":89,"thread_start_us":17359,"threads_started":1}
I20260812 02:36:05.004218 8998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:04.977021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53088 (local address 127.7.132.131:37211)
0812 02:36:04.990341 (+ 13320us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:04.990349 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:04.994474 (+ 4125us) server_negotiation.cc:407] Connection header received
0812 02:36:04.994597 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:04.994629 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:04.994874 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:04.995096 (+ 222us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:36:04.995918 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:04.997430 (+ 1512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:05.001508 (+ 4078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:05.003164 (+ 1656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:05.003324 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:05.003560 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:05.003650 (+ 90us) server_negotiation.cc:299] Negotiation successful
0812 02:36:05.003814 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13129,"thread_start_us":13312,"threads_started":1}
I20260812 02:36:05.004809 8736 tablet_service.cc:1467] Processing CreateTablet for tablet f6a4590b39cc48bd8b9135a5db994366 (DEFAULT_TABLE table=test-table [id=b032fecd7385410babf7b8f904f9102a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:36:05.006583 8736 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f6a4590b39cc48bd8b9135a5db994366. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:36:05.008219 8891 tablet_service.cc:1467] Processing CreateTablet for tablet f6a4590b39cc48bd8b9135a5db994366 (DEFAULT_TABLE table=test-table [id=b032fecd7385410babf7b8f904f9102a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:36:05.009922 8891 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f6a4590b39cc48bd8b9135a5db994366. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:36:05.009665 8565 tablet_service.cc:1467] Processing CreateTablet for tablet f6a4590b39cc48bd8b9135a5db994366 (DEFAULT_TABLE table=test-table [id=b032fecd7385410babf7b8f904f9102a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:36:05.010859 8565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f6a4590b39cc48bd8b9135a5db994366. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:36:05.022802 8999 tablet_bootstrap.cc:492] T f6a4590b39cc48bd8b9135a5db994366 P f80db53180114a2f9b110d710696a857: Bootstrap starting.
I20260812 02:36:05.026402 8999 tablet_bootstrap.cc:654] T f6a4590b39cc48bd8b9135a5db994366 P f80db53180114a2f9b110d710696a857: Neither blocks nor log segments found. Creating new log.
I20260812 02:36:05.026832 9002 tablet_bootstrap.cc:492] T f6a4590b39cc48bd8b9135a5db994366 P 4579e6b2dea1461dbd4c22e57ee97f2d: Bootstrap starting.
I20260812 02:36:05.027932 8999 log.cc:824] T f6a4590b39cc48bd8b9135a5db994366 P f80db53180114a2f9b110d710696a857: Log is configured to *not* fsync() on all Append() calls
I20260812 02:36:05.029069 9000 tablet_bootstrap.cc:492] T f6a4590b39cc48bd8b9135a5db994366 P 2ae594811b024a34aff5ea4109c4c586: Bootstrap starting.
I20260812 02:36:05.030895 9002 tablet_bootstrap.cc:654] T f6a4590b39cc48bd8b9135a5db994366 P 4579e6b2dea1461dbd4c22e57ee97f2d: Neither blocks nor log segments found. Creating new log.
I20260812 02:36:05.032063 9002 log.cc:824] T f6a4590b39cc48bd8b9135a5db994366 P 4579e6b2dea1461dbd4c22e57ee97f2d: Log is configured to *not* fsync() on all Append() calls
I20260812 02:36:05.032868 9000 tablet_bootstrap.cc:654] T f6a4590b39cc48bd8b9135a5db994366 P 2ae594811b024a34aff5ea4109c4c586: Neither blocks nor log segments found. Creating new log.
I20260812 02:36:05.033802 8999 tablet_bootstrap.cc:492] T f6a4590b39cc48bd8b9135a5db994366 P f80db53180114a2f9b110d710696a857: No bootstrap required, opened a new log
I20260812 02:36:05.034119 8999 ts_tablet_manager.cc:1397] T f6a4590b39cc48bd8b9135a5db994366 P f80db53180114a2f9b110d710696a857: Time spent bootstrapping tablet: real 0.012s user 0.008s sys 0.000s
I20260812 02:36:05.034258 9000 log.cc:824] T f6a4590b39cc48bd8b9135a5db994366 P 2ae594811b024a34aff5ea4109c4c586: Log is configured to *not* fsync() on all Append() calls
I20260812 02:36:05.035389 9002 tablet_bootstrap.cc:492] T f6a4590b39cc48bd8b9135a5db994366 P 4579e6b2dea1461dbd4c22e57ee97f2d: No bootstrap required, opened a new log
I20260812 02:36:05.035671 9002 ts_tablet_manager.cc:1397] T f6a4590b39cc48bd8b9135a5db994366 P 4579e6b2dea1461dbd4c22e57ee97f2d: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.003s
I20260812 02:36:05.037065 9000 tablet_bootstrap.cc:492] T f6a4590b39cc48bd8b9135a5db994366 P 2ae594811b024a34aff5ea4109c4c586: No bootstrap required, opened a new log
I20260812 02:36:05.037335 9000 ts_tablet_manager.cc:1397] T f6a4590b39cc48bd8b9135a5db994366 P 2ae594811b024a34aff5ea4109c4c586: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260812 02:36:05.045109 8999 raft_consensus.cc:348] T f6a4590b39cc48bd8b9135a5db994366 P f80db53180114a2f9b110d710696a857 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ae594811b024a34aff5ea4109c4c586" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 43411 } } peers { permanent_uuid: "f80db53180114a2f9b110d710696a857" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 40903 } } peers { permanent_uuid: "4579e6b2dea1461dbd4c22e57ee97f2d" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 37211 } }
I20260812 02:36:05.045575 8999 raft_consensus.cc:374] T f6a4590b39cc48bd8b9135a5db994366 P f80db53180114a2f9b110d710696a857 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:36:05.045346 9000 raft_consensus.cc:348] T f6a4590b39cc48bd8b9135a5db994366 P 2ae594811b024a34aff5ea4109c4c586 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ae594811b024a34aff5ea4109c4c586" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 43411 } } peers { permanent_uuid: "f80db53180114a2f9b110d710696a857" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 40903 } } peers { permanent_uuid: "4579e6b2dea1461dbd4c22e57ee97f2d" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 37211 } }
I20260812 02:36:05.045673 8999 raft_consensus.cc:729] T f6a4590b39cc48bd8b9135a5db994366 P f80db53180114a2f9b110d710696a857 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f80db53180114a2f9b110d710696a857, State: Initialized, Role: FOLLOWER
I20260812 02:36:05.045825 9000 raft_consensus.cc:374] T f6a4590b39cc48bd8b9135a5db994366 P 2ae594811b024a34aff5ea4109c4c586 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:36:05.045951 9000 raft_consensus.cc:729] T f6a4590b39cc48bd8b9135a5db994366 P 2ae594811b024a34aff5ea4109c4c586 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ae594811b024a34aff5ea4109c4c586, State: Initialized, Role: FOLLOWER
I20260812 02:36:05.046344 8999 consensus_queue.cc:260] T f6a4590b39cc48bd8b9135a5db994366 P f80db53180114a2f9b110d710696a857 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ae594811b024a34aff5ea4109c4c586" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 43411 } } peers { permanent_uuid: "f80db53180114a2f9b110d710696a857" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 40903 } } peers { permanent_uuid: "4579e6b2dea1461dbd4c22e57ee97f2d" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 37211 } }
I20260812 02:36:05.046368 9002 raft_consensus.cc:348] T f6a4590b39cc48bd8b9135a5db994366 P 4579e6b2dea1461dbd4c22e57ee97f2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ae594811b024a34aff5ea4109c4c586" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 43411 } } peers { permanent_uuid: "f80db53180114a2f9b110d710696a857" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 40903 } } peers { permanent_uuid: "4579e6b2dea1461dbd4c22e57ee97f2d" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 37211 } }
I20260812 02:36:05.046789 9002 raft_consensus.cc:374] T f6a4590b39cc48bd8b9135a5db994366 P 4579e6b2dea1461dbd4c22e57ee97f2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:36:05.046630 9000 consensus_queue.cc:260] T f6a4590b39cc48bd8b9135a5db994366 P 2ae594811b024a34aff5ea4109c4c586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ae594811b024a34aff5ea4109c4c586" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 43411 } } peers { permanent_uuid: "f80db53180114a2f9b110d710696a857" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 40903 } } peers { permanent_uuid: "4579e6b2dea1461dbd4c22e57ee97f2d" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 37211 } }
I20260812 02:36:05.046921 9002 raft_consensus.cc:729] T f6a4590b39cc48bd8b9135a5db994366 P 4579e6b2dea1461dbd4c22e57ee97f2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4579e6b2dea1461dbd4c22e57ee97f2d, State: Initialized, Role: FOLLOWER
I20260812 02:36:05.047641 9002 consensus_queue.cc:260] T f6a4590b39cc48bd8b9135a5db994366 P 4579e6b2dea1461dbd4c22e57ee97f2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ae594811b024a34aff5ea4109c4c586" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 43411 } } peers { permanent_uuid: "f80db53180114a2f9b110d710696a857" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 40903 } } peers { permanent_uuid: "4579e6b2dea1461dbd4c22e57ee97f2d" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 37211 } }
I20260812 02:36:05.048449 8812 heartbeater.cc:503] Master 127.7.132.190:34915 was elected leader, sending a full tablet report...
I20260812 02:36:05.049142 9000 ts_tablet_manager.cc:1428] T f6a4590b39cc48bd8b9135a5db994366 P 2ae594811b024a34aff5ea4109c4c586: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
I20260812 02:36:05.050595 8999 ts_tablet_manager.cc:1428] T f6a4590b39cc48bd8b9135a5db994366 P f80db53180114a2f9b110d710696a857: Time spent starting tablet: real 0.016s user 0.012s sys 0.002s
I20260812 02:36:05.051640 8974 heartbeater.cc:503] Master 127.7.132.190:34915 was elected leader, sending a full tablet report...
I20260812 02:36:05.052188 9002 ts_tablet_manager.cc:1428] T f6a4590b39cc48bd8b9135a5db994366 P 4579e6b2dea1461dbd4c22e57ee97f2d: Time spent starting tablet: real 0.016s user 0.013s sys 0.004s
W20260812 02:36:05.094622 8646 tablet.cc:2369] T f6a4590b39cc48bd8b9135a5db994366 P f80db53180114a2f9b110d710696a857: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 02:36:05.112437 8814 tablet.cc:2369] T f6a4590b39cc48bd8b9135a5db994366 P 2ae594811b024a34aff5ea4109c4c586: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 02:36:05.144212 8976 tablet.cc:2369] T f6a4590b39cc48bd8b9135a5db994366 P 4579e6b2dea1461dbd4c22e57ee97f2d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 02:36:05.424824 9009 raft_consensus.cc:482] T f6a4590b39cc48bd8b9135a5db994366 P 4579e6b2dea1461dbd4c22e57ee97f2d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 02:36:05.425148 9009 raft_consensus.cc:504] T f6a4590b39cc48bd8b9135a5db994366 P 4579e6b2dea1461dbd4c22e57ee97f2d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ae594811b024a34aff5ea4109c4c586" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 43411 } } peers { permanent_uuid: "f80db53180114a2f9b110d710696a857" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 40903 } } peers { permanent_uuid: "4579e6b2dea1461dbd4c22e57ee97f2d" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 37211 } }
I20260812 02:36:05.426893 9009 leader_election.cc:288] T f6a4590b39cc48bd8b9135a5db994366 P 4579e6b2dea1461dbd4c22e57ee97f2d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2ae594811b024a34aff5ea4109c4c586 (127.7.132.130:43411), f80db53180114a2f9b110d710696a857 (127.7.132.129:40903)
I20260812 02:36:05.434163 8991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:05.427614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:60139 (local address 127.7.132.130:43411)
0812 02:36:05.427853 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:05.427860 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:05.428265 (+ 405us) server_negotiation.cc:407] Connection header received
0812 02:36:05.428489 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:05.428494 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:05.428566 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:05.428670 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:36:05.429591 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:05.430525 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:05.432321 (+ 1796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:05.433152 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:05.433336 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:05.433411 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:05.433700 (+ 289us) server_negotiation.cc:299] Negotiation successful
0812 02:36:05.433831 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 02:36:05.434188 8997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:05.427466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:43199 (local address 127.7.132.129:40903)
0812 02:36:05.427695 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:05.427703 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:05.428274 (+ 571us) server_negotiation.cc:407] Connection header received
0812 02:36:05.428476 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:05.428481 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:05.428566 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:05.428671 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:36:05.429671 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:05.430587 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:05.432371 (+ 1784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:05.433184 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:05.433378 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:05.433453 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:05.433748 (+ 295us) server_negotiation.cc:299] Negotiation successful
0812 02:36:05.433886 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260812 02:36:05.434302 9011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:05.427239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.130:43411 (local address 127.7.132.131:60139)
0812 02:36:05.428105 (+ 866us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:05.428126 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:05.428320 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:05.428709 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:05.428714 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:05.428752 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:36:05.429416 (+ 664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:05.429435 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:05.430668 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:05.430673 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:05.432158 (+ 1485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:05.432170 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:05.433320 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:05.433325 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:05.433540 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:05.433565 (+ 25us) client_negotiation.cc:769] Sending connection context
0812 02:36:05.433674 (+ 109us) client_negotiation.cc:240] Negotiation successful
0812 02:36:05.433831 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":732,"mutex_wait_us":30,"spinlock_wait_cycles":25984,"thread_start_us":603,"threads_started":1}
I20260812 02:36:05.434726 9010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:05.427239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:40903 (local address 127.7.132.131:43199)
0812 02:36:05.428159 (+ 920us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:05.428175 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:05.428320 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:05.428709 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:05.428714 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:05.428749 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:36:05.429489 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:05.429503 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:05.430721 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:05.430730 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:05.432204 (+ 1474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:05.432216 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:05.433414 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:05.433420 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:05.433589 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:05.433613 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 02:36:05.433718 (+ 105us) client_negotiation.cc:240] Negotiation successful
0812 02:36:05.433881 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":812,"mutex_wait_us":147,"spinlock_wait_cycles":30720,"thread_start_us":486,"threads_started":1}
I20260812 02:36:05.435348 8758 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6a4590b39cc48bd8b9135a5db994366" candidate_uuid: "4579e6b2dea1461dbd4c22e57ee97f2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ae594811b024a34aff5ea4109c4c586" is_pre_election: true
I20260812 02:36:05.435746 8590 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6a4590b39cc48bd8b9135a5db994366" candidate_uuid: "4579e6b2dea1461dbd4c22e57ee97f2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f80db53180114a2f9b110d710696a857" is_pre_election: true
I20260812 02:36:05.435881 8758 raft_consensus.cc:2454] T f6a4590b39cc48bd8b9135a5db994366 P 2ae594811b024a34aff5ea4109c4c586 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4579e6b2dea1461dbd4c22e57ee97f2d in term 0.
I20260812 02:36:05.436234 8590 raft_consensus.cc:2454] T f6a4590b39cc48bd8b9135a5db994366 P f80db53180114a2f9b110d710696a857 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4579e6b2dea1461dbd4c22e57ee97f2d in term 0.
I20260812 02:36:05.436583 8838 leader_election.cc:302] T f6a4590b39cc48bd8b9135a5db994366 P 4579e6b2dea1461dbd4c22e57ee97f2d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ae594811b024a34aff5ea4109c4c586, 4579e6b2dea1461dbd4c22e57ee97f2d; no voters:
I20260812 02:36:05.437026 9009 raft_consensus.cc:2781] T f6a4590b39cc48bd8b9135a5db994366 P 4579e6b2dea1461dbd4c22e57ee97f2d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 02:36:05.437193 9009 raft_consensus.cc:482] T f6a4590b39cc48bd8b9135a5db994366 P 4579e6b2dea1461dbd4c22e57ee97f2d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 02:36:05.437309 9009 raft_consensus.cc:3037] T f6a4590b39cc48bd8b9135a5db994366 P 4579e6b2dea1461dbd4c22e57ee97f2d [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:36:05.439677 9009 raft_consensus.cc:504] T f6a4590b39cc48bd8b9135a5db994366 P 4579e6b2dea1461dbd4c22e57ee97f2d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ae594811b024a34aff5ea4109c4c586" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 43411 } } peers { permanent_uuid: "f80db53180114a2f9b110d710696a857" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 40903 } } peers { permanent_uuid: "4579e6b2dea1461dbd4c22e57ee97f2d" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 37211 } }
I20260812 02:36:05.440272 9009 leader_election.cc:288] T f6a4590b39cc48bd8b9135a5db994366 P 4579e6b2dea1461dbd4c22e57ee97f2d [CANDIDATE]: Term 1 election: Requested vote from peers 2ae594811b024a34aff5ea4109c4c586 (127.7.132.130:43411), f80db53180114a2f9b110d710696a857 (127.7.132.129:40903)
I20260812 02:36:05.440742 8758 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6a4590b39cc48bd8b9135a5db994366" candidate_uuid: "4579e6b2dea1461dbd4c22e57ee97f2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ae594811b024a34aff5ea4109c4c586"
I20260812 02:36:05.440768 8590 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6a4590b39cc48bd8b9135a5db994366" candidate_uuid: "4579e6b2dea1461dbd4c22e57ee97f2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f80db53180114a2f9b110d710696a857"
I20260812 02:36:05.440937 8758 raft_consensus.cc:3037] T f6a4590b39cc48bd8b9135a5db994366 P 2ae594811b024a34aff5ea4109c4c586 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:36:05.440968 8590 raft_consensus.cc:3037] T f6a4590b39cc48bd8b9135a5db994366 P f80db53180114a2f9b110d710696a857 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:36:05.443704 8590 raft_consensus.cc:2454] T f6a4590b39cc48bd8b9135a5db994366 P f80db53180114a2f9b110d710696a857 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4579e6b2dea1461dbd4c22e57ee97f2d in term 1.
I20260812 02:36:05.443708 8758 raft_consensus.cc:2454] T f6a4590b39cc48bd8b9135a5db994366 P 2ae594811b024a34aff5ea4109c4c586 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4579e6b2dea1461dbd4c22e57ee97f2d in term 1.
I20260812 02:36:05.444198 8839 leader_election.cc:302] T f6a4590b39cc48bd8b9135a5db994366 P 4579e6b2dea1461dbd4c22e57ee97f2d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2ae594811b024a34aff5ea4109c4c586, 4579e6b2dea1461dbd4c22e57ee97f2d, f80db53180114a2f9b110d710696a857; no voters:
I20260812 02:36:05.444568 9009 raft_consensus.cc:2781] T f6a4590b39cc48bd8b9135a5db994366 P 4579e6b2dea1461dbd4c22e57ee97f2d [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:36:05.445207 9009 raft_consensus.cc:686] T f6a4590b39cc48bd8b9135a5db994366 P 4579e6b2dea1461dbd4c22e57ee97f2d [term 1 LEADER]: Becoming Leader. State: Replica: 4579e6b2dea1461dbd4c22e57ee97f2d, State: Running, Role: LEADER
I20260812 02:36:05.445749 9009 consensus_queue.cc:237] T f6a4590b39cc48bd8b9135a5db994366 P 4579e6b2dea1461dbd4c22e57ee97f2d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ae594811b024a34aff5ea4109c4c586" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 43411 } } peers { permanent_uuid: "f80db53180114a2f9b110d710696a857" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 40903 } } peers { permanent_uuid: "4579e6b2dea1461dbd4c22e57ee97f2d" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 37211 } }
I20260812 02:36:05.450735 8406 catalog_manager.cc:5495] T f6a4590b39cc48bd8b9135a5db994366 P 4579e6b2dea1461dbd4c22e57ee97f2d reported cstate change: term changed from 0 to 1, leader changed from <none> to 4579e6b2dea1461dbd4c22e57ee97f2d (127.7.132.131). New cstate: current_term: 1 leader_uuid: "4579e6b2dea1461dbd4c22e57ee97f2d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ae594811b024a34aff5ea4109c4c586" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 43411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f80db53180114a2f9b110d710696a857" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4579e6b2dea1461dbd4c22e57ee97f2d" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 37211 } health_report { overall_health: HEALTHY } } }
Aug 12 02:36:05 dist-test-slave-mn4r krb5kdc[8350](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502162, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.132.129@KRBTEST.COM
I20260812 02:36:05.570659 8997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:05.555763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43414 (local address 127.7.132.129:40903)
0812 02:36:05.555968 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:05.555976 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:05.556365 (+ 389us) server_negotiation.cc:407] Connection header received
0812 02:36:05.556600 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:05.556604 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:05.556674 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:05.556785 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:05.558222 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:05.559023 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:05.560320 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:05.560721 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:05.560867 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:05.565046 (+ 4179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:05.565061 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:05.565077 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:05.565120 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:05.568122 (+ 3002us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:05.568757 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:05.568764 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:05.568766 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:05.568899 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:05.569230 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:05.569233 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:05.569235 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:05.569568 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:05.569797 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:05.570035 (+ 238us) server_negotiation.cc:299] Negotiation successful
0812 02:36:05.570276 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
Aug 12 02:36:05 dist-test-slave-mn4r krb5kdc[8350](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502165, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 02:36:05 dist-test-slave-mn4r krb5kdc[8350](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502165, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.132.129@KRBTEST.COM
I20260812 02:36:05.620730 8997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:05.606673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43422 (local address 127.7.132.129:40903)
0812 02:36:05.606886 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:05.606894 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:05.607273 (+ 379us) server_negotiation.cc:407] Connection header received
0812 02:36:05.607495 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:05.607500 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:05.607567 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:05.607676 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:05.609094 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:05.609844 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:05.611154 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:05.611569 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:05.611709 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:05.615639 (+ 3930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:05.615645 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:05.615648 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:05.615681 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:05.618200 (+ 2519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:05.618808 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:05.618814 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:05.618817 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:05.618948 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:05.619352 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:05.619358 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:05.619361 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:05.619609 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:05.619875 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:05.620165 (+ 290us) server_negotiation.cc:299] Negotiation successful
0812 02:36:05.620385 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
Aug 12 02:36:05 dist-test-slave-mn4r krb5kdc[8350](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502165, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 02:36:05 dist-test-slave-mn4r krb5kdc[8350](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502165, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.132.129@KRBTEST.COM
I20260812 02:36:05.669641 8997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:05.655601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43438 (local address 127.7.132.129:40903)
0812 02:36:05.655814 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:05.655822 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:05.656170 (+ 348us) server_negotiation.cc:407] Connection header received
0812 02:36:05.656393 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:05.656397 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:05.656463 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:05.656582 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:05.657934 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:05.658881 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:05.660114 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:05.660553 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:05.660658 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:05.664582 (+ 3924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:05.664587 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:05.664590 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:05.664624 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:05.667379 (+ 2755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:05.668037 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:05.668041 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:05.668043 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:05.668161 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:05.668471 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:05.668474 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:05.668476 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:05.668741 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:05.668930 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:05.669184 (+ 254us) server_negotiation.cc:299] Negotiation successful
0812 02:36:05.669353 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
W20260812 02:36:05.671368 8535 tablet_service.cc:2908] Rejecting scan request for tablet f6a4590b39cc48bd8b9135a5db994366: Uninitialized: clean time has not yet been initialized
I20260812 02:36:05.674361 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 8484
I20260812 02:36:05.686350 8997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:05.675242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43440 (local address 127.7.132.129:40903)
0812 02:36:05.675431 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:05.675435 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:05.675868 (+ 433us) server_negotiation.cc:407] Connection header received
0812 02:36:05.675984 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:05.675987 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:05.676070 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:05.676146 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:05.677250 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:05.677911 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:05.679095 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:05.679476 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:05.679566 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:05.681707 (+ 2141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:05.681712 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:05.681715 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:05.681743 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:05.684088 (+ 2345us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:05.684655 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:05.684658 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:05.684661 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:05.684766 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:05.685084 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:05.685091 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:05.685095 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:05.685429 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:05.685647 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:05.685861 (+ 214us) server_negotiation.cc:299] Negotiation successful
0812 02:36:05.686043 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 02:36:05.686986 8638 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:36:05.777290 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 8484
I20260812 02:36:05.799865 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 8651
Aug 12 02:36:05 dist-test-slave-mn4r krb5kdc[8350](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502165, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.132.130@KRBTEST.COM
I20260812 02:36:05.815182 8991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:05.800648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60698 (local address 127.7.132.130:43411)
0812 02:36:05.800838 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:05.800845 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:05.800910 (+ 65us) server_negotiation.cc:407] Connection header received
0812 02:36:05.801133 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:05.801139 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:05.801222 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:05.801330 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:05.802737 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:05.803444 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:05.804709 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:05.805119 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:05.805257 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:05.809464 (+ 4207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:05.809484 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:05.809500 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:05.809536 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:05.812310 (+ 2774us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:05.813041 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:05.813050 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:05.813053 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:05.813185 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:05.813580 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:05.813586 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:05.813589 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:05.813964 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:05.814320 (+ 356us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:05.814604 (+ 284us) server_negotiation.cc:299] Negotiation successful
0812 02:36:05.814826 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260812 02:36:05.815980 8806 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:36:05.905625 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 8651
W20260812 02:36:05.926925 8838 connection.cc:535] client connection to 127.7.132.130:43411 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260812 02:36:05.927265 8838 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 02:36:05.927533 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 8819
I20260812 02:36:05.929440 9010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:05.929033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.130:43411 (local address 127.7.132.131:37625)
0812 02:36:05.929192 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:05.929304 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.130:43411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260812 02:36:05.929628 8838 consensus_peers.cc:487] T f6a4590b39cc48bd8b9135a5db994366 P 4579e6b2dea1461dbd4c22e57ee97f2d -> Peer 2ae594811b024a34aff5ea4109c4c586 (127.7.132.130:43411): Couldn't send request to peer 2ae594811b024a34aff5ea4109c4c586. Status: Network error: Client connection negotiation failed: client connection to 127.7.132.130:43411: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
Aug 12 02:36:05 dist-test-slave-mn4r krb5kdc[8350](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502165, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.132.131@KRBTEST.COM
I20260812 02:36:05.943223 9035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:05.928379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53096 (local address 127.7.132.131:37211)
0812 02:36:05.928879 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:05.928884 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:05.928913 (+ 29us) server_negotiation.cc:407] Connection header received
0812 02:36:05.928974 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:05.928977 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:05.929038 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:05.929177 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:05.930693 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:05.931451 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:05.932749 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:05.933182 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:05.933327 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:05.937494 (+ 4167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:05.937506 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:05.937521 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:05.937553 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:05.940263 (+ 2710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:05.940936 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:05.940943 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:05.940946 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:05.941076 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:05.941466 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:05.941472 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:05.941475 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:05.941795 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:05.942155 (+ 360us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:05.942635 (+ 480us) server_negotiation.cc:299] Negotiation successful
0812 02:36:05.942870 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":382,"thread_start_us":276,"threads_started":1}
I20260812 02:36:05.943890 8968 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:36:06.033661 9010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:06.033133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:40903 (local address 127.7.132.131:33497)
0812 02:36:06.033309 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:06.033455 (+ 146us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.129:40903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260812 02:36:06.033691 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 8819
I20260812 02:36:06.055090 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 8364
I20260812 02:36:06.056308 8443 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:36:06.166381 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 8364
2026-08-12T02:36:06Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (4819 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502156971023-7698-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 02:36:06 dist-test-slave-mn4r krb5kdc[9040](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 02:36:06 dist-test-slave-mn4r krb5kdc[9040](info): set up 2 sockets
Aug 12 02:36:06 dist-test-slave-mn4r krb5kdc[9040](info): commencing operation
krb5kdc: starting...
W20260812 02:36:07.350257 7698 mini_kdc.cc:121] Time spent starting KDC: real 1.136s user 0.007s sys 0.013s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502156971023-7698-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502156971023-7698-0/krb5kdc/test-user.keytab.
Aug 12 02:36:07 dist-test-slave-mn4r krb5kdc[9040](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502167, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T02:36:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T02:36:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.132.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
I20260812 02:36:07.573915 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502156971023-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502156971023-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502156971023-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:43261
--webserver_interface=127.7.132.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:43895
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.132.190:43261
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:36:07.899798 9056 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:36:07.900101 9056 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:36:07.900192 9056 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:36:07.907456 9056 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:36:07.907563 9056 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260812 02:36:07.907604 9056 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:36:07.907665 9056 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:36:07.907716 9056 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:36:07.907770 9056 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:36:07.917529 9056 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:43895
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502156971023-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502156971023-7698-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.132.190:43261
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:43261
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.132.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502156971023-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:36:07.918855 9056 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:36:07.920117 9056 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:36:07.934571 9063 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:36:07.934599 9066 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:36:07.934603 9064 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:36:07.936087 9056 server_base.cc:1034] running on GCE node
I20260812 02:36:07.936865 9056 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:36:07.938323 9056 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:36:07.939575 9056 hybrid_clock.cc:648] HybridClock initialized: now 1786502167939509 us; error 40 us; skew 500 ppm
I20260812 02:36:07.939906 9056 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:36:07 dist-test-slave-mn4r krb5kdc[9040](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502167, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:36:07.943120 9056 init.cc:377] Logged in from keytab as kudu/127.7.132.190@KRBTEST.COM (short username kudu)
I20260812 02:36:07.944655 9056 webserver.cc:460] Webserver started at http://127.7.132.190:41635/ using document root <none> and password file <none>
I20260812 02:36:07.945370 9056 fs_manager.cc:359] Metadata directory not provided
I20260812 02:36:07.945505 9056 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:36:07.945842 9056 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:36:07.948892 9056 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502156971023-7698-0/minicluster-data/master-0/data/instance:
uuid: "38e2dd3af22545f6b021313a489f7387"
format_stamp: "Formatted at 2026-08-12 02:36:07 on dist-test-slave-mn4r"
server_key: "0e4a00bd53b511d4cc23c1d7612cff26"
server_key_iv: "7cfc125534d679c5b87887bc8b21a9c5"
server_key_version: "encryptionkey@0"
I20260812 02:36:07.949719 9056 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502156971023-7698-0/minicluster-data/master-0/wal/instance:
uuid: "38e2dd3af22545f6b021313a489f7387"
format_stamp: "Formatted at 2026-08-12 02:36:07 on dist-test-slave-mn4r"
server_key: "0e4a00bd53b511d4cc23c1d7612cff26"
server_key_iv: "7cfc125534d679c5b87887bc8b21a9c5"
server_key_version: "encryptionkey@0"
I20260812 02:36:07.955001 9056 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260812 02:36:07.958405 9074 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:36:07.959664 9056 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 02:36:07.959853 9056 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502156971023-7698-0/minicluster-data/master-0/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502156971023-7698-0/minicluster-data/master-0/wal
uuid: "38e2dd3af22545f6b021313a489f7387"
format_stamp: "Formatted at 2026-08-12 02:36:07 on dist-test-slave-mn4r"
server_key: "0e4a00bd53b511d4cc23c1d7612cff26"
server_key_iv: "7cfc125534d679c5b87887bc8b21a9c5"
server_key_version: "encryptionkey@0"
I20260812 02:36:07.960090 9056 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502156971023-7698-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502156971023-7698-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502156971023-7698-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:36:07.976217 9056 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:36:07.980387 9056 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:36:07.980675 9056 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:36:08.003520 9056 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.190:43261
I20260812 02:36:08.003538 9154 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.190:43261 every 8 connection(s)
I20260812 02:36:08.004969 9056 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
I20260812 02:36:08.008860 9155 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:36:08.012665 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 9056
I20260812 02:36:08.012897 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502156971023-7698-0/minicluster-data/master-0/wal/instance
I20260812 02:36:08.013684 7698 external_mini_cluster.cc:1428] Setting key 24602a97799f3bfee609ebfd4b06d50c
I20260812 02:36:08.022804 9155 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38e2dd3af22545f6b021313a489f7387: Bootstrap starting.
I20260812 02:36:08.026723 9155 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 38e2dd3af22545f6b021313a489f7387: Neither blocks nor log segments found. Creating new log.
I20260812 02:36:08.028100 9155 log.cc:824] T 00000000000000000000000000000000 P 38e2dd3af22545f6b021313a489f7387: Log is configured to *not* fsync() on all Append() calls
Aug 12 02:36:08 dist-test-slave-mn4r krb5kdc[9040](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502167, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 02:36:08.031327 9155 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38e2dd3af22545f6b021313a489f7387: No bootstrap required, opened a new log
I20260812 02:36:08.037758 9158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:08.015908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50270 (local address 127.7.132.190:43261)
0812 02:36:08.016724 (+ 816us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:08.016736 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:08.016778 (+ 42us) server_negotiation.cc:407] Connection header received
0812 02:36:08.018496 (+ 1718us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:08.018524 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:08.018923 (+ 399us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:08.019266 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:08.021102 (+ 1836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:08.022508 (+ 1406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:08.024244 (+ 1736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:08.024952 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:08.025088 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:08.030294 (+ 5206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:08.030306 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:08.030327 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:08.030374 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:08.033620 (+ 3246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:08.034484 (+ 864us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:08.034490 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:08.034497 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:08.034665 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:08.035063 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:08.035067 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:08.035069 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:08.035582 (+ 513us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:08.035831 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:08.036277 (+ 446us) server_negotiation.cc:299] Negotiation successful
0812 02:36:08.036658 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":597,"thread_start_us":380,"threads_started":1}
I20260812 02:36:08.040498 9155 raft_consensus.cc:348] T 00000000000000000000000000000000 P 38e2dd3af22545f6b021313a489f7387 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38e2dd3af22545f6b021313a489f7387" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 43261 } }
I20260812 02:36:08.040966 9155 raft_consensus.cc:374] T 00000000000000000000000000000000 P 38e2dd3af22545f6b021313a489f7387 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:36:08.041082 9155 raft_consensus.cc:729] T 00000000000000000000000000000000 P 38e2dd3af22545f6b021313a489f7387 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38e2dd3af22545f6b021313a489f7387, State: Initialized, Role: FOLLOWER
I20260812 02:36:08.041675 9155 consensus_queue.cc:260] T 00000000000000000000000000000000 P 38e2dd3af22545f6b021313a489f7387 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38e2dd3af22545f6b021313a489f7387" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 43261 } }
I20260812 02:36:08.041941 9155 raft_consensus.cc:388] T 00000000000000000000000000000000 P 38e2dd3af22545f6b021313a489f7387 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 02:36:08.042122 9155 raft_consensus.cc:482] T 00000000000000000000000000000000 P 38e2dd3af22545f6b021313a489f7387 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 02:36:08.042311 9155 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 38e2dd3af22545f6b021313a489f7387 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:36:08.044413 9155 raft_consensus.cc:504] T 00000000000000000000000000000000 P 38e2dd3af22545f6b021313a489f7387 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38e2dd3af22545f6b021313a489f7387" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 43261 } }
I20260812 02:36:08.044909 9155 leader_election.cc:302] T 00000000000000000000000000000000 P 38e2dd3af22545f6b021313a489f7387 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 38e2dd3af22545f6b021313a489f7387; no voters:
I20260812 02:36:08.045540 9155 leader_election.cc:288] T 00000000000000000000000000000000 P 38e2dd3af22545f6b021313a489f7387 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 02:36:08.045665 9162 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 38e2dd3af22545f6b021313a489f7387 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:36:08.046555 9162 raft_consensus.cc:686] T 00000000000000000000000000000000 P 38e2dd3af22545f6b021313a489f7387 [term 1 LEADER]: Becoming Leader. State: Replica: 38e2dd3af22545f6b021313a489f7387, State: Running, Role: LEADER
I20260812 02:36:08.047135 9162 consensus_queue.cc:237] T 00000000000000000000000000000000 P 38e2dd3af22545f6b021313a489f7387 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38e2dd3af22545f6b021313a489f7387" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 43261 } }
I20260812 02:36:08.047573 9155 sys_catalog.cc:564] T 00000000000000000000000000000000 P 38e2dd3af22545f6b021313a489f7387 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:36:08.050133 9164 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38e2dd3af22545f6b021313a489f7387 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 38e2dd3af22545f6b021313a489f7387. Latest consensus state: current_term: 1 leader_uuid: "38e2dd3af22545f6b021313a489f7387" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38e2dd3af22545f6b021313a489f7387" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 43261 } } }
I20260812 02:36:08.049870 9163 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38e2dd3af22545f6b021313a489f7387 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "38e2dd3af22545f6b021313a489f7387" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38e2dd3af22545f6b021313a489f7387" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 43261 } } }
I20260812 02:36:08.050562 9164 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38e2dd3af22545f6b021313a489f7387 [sys.catalog]: This master's current role is: LEADER
I20260812 02:36:08.050575 9163 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38e2dd3af22545f6b021313a489f7387 [sys.catalog]: This master's current role is: LEADER
I20260812 02:36:08.051600 9170 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:36:08.058637 9170 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 02:36:08.067646 9170 catalog_manager.cc:1347] Generated new cluster ID: 044f0894d81c48dda1f762e90f15f36b
I20260812 02:36:08.067764 9170 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 02:36:08.105744 9170 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 02:36:08.107741 9170 catalog_manager.cc:1504] Loading token signing keys...
I20260812 02:36:08.118952 9170 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 38e2dd3af22545f6b021313a489f7387: Generated new TSK 0
I20260812 02:36:08.120002 9170 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.132.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.129.keytab.
Entry for principal kudu/127.7.132.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.129.keytab.
I20260812 02:36:08.173627 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502156971023-7698-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502156971023-7698-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502156971023-7698-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502156971023-7698-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.129:0
--local_ip_for_outbound_sockets=127.7.132.129
--webserver_interface=127.7.132.129
--webserver_port=0
--tserver_master_addrs=127.7.132.190:43261
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.129.keytab
--principal=kudu/127.7.132.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:43895
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:36:08.516026 9187 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:36:08.516314 9187 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:36:08.516409 9187 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:36:08.523722 9187 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:36:08.523835 9187 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:36:08.523957 9187 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 02:36:08.524042 9187 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.129
I20260812 02:36:08.534129 9187 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:43895
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502156971023-7698-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502156971023-7698-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.129.keytab
--principal=kudu/127.7.132.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502156971023-7698-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.132.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.132.190:43261
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.129
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502156971023-7698-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:36:08.535507 9187 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:36:08.536834 9187 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:36:08.550359 9193 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:36:08.550398 9194 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:36:08.550302 9196 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:36:08.551771 9187 server_base.cc:1034] running on GCE node
I20260812 02:36:08.552414 9187 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:36:08.553387 9187 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:36:08.554633 9187 hybrid_clock.cc:648] HybridClock initialized: now 1786502168554573 us; error 36 us; skew 500 ppm
I20260812 02:36:08.554939 9187 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:36:08 dist-test-slave-mn4r krb5kdc[9040](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502168, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:36:08.563097 9187 init.cc:377] Logged in from keytab as kudu/127.7.132.129@KRBTEST.COM (short username kudu)
I20260812 02:36:08.564814 9187 webserver.cc:460] Webserver started at http://127.7.132.129:40129/ using document root <none> and password file <none>
I20260812 02:36:08.565562 9187 fs_manager.cc:359] Metadata directory not provided
I20260812 02:36:08.565697 9187 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:36:08.566043 9187 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:36:08.569121 9187 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502156971023-7698-0/minicluster-data/ts-0/data/instance:
uuid: "3c7b0dc3dde34ef98a501ea15c294a1b"
format_stamp: "Formatted at 2026-08-12 02:36:08 on dist-test-slave-mn4r"
server_key: "c3773998d981356b2fa4c71ec8b9539c"
server_key_iv: "fbeb5291545b5468b1462446f001d28b"
server_key_version: "encryptionkey@0"
I20260812 02:36:08.569943 9187 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502156971023-7698-0/minicluster-data/ts-0/wal/instance:
uuid: "3c7b0dc3dde34ef98a501ea15c294a1b"
format_stamp: "Formatted at 2026-08-12 02:36:08 on dist-test-slave-mn4r"
server_key: "c3773998d981356b2fa4c71ec8b9539c"
server_key_iv: "fbeb5291545b5468b1462446f001d28b"
server_key_version: "encryptionkey@0"
I20260812 02:36:08.575409 9187 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260812 02:36:08.579048 9204 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:36:08.580400 9187 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 02:36:08.580590 9187 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502156971023-7698-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502156971023-7698-0/minicluster-data/ts-0/wal
uuid: "3c7b0dc3dde34ef98a501ea15c294a1b"
format_stamp: "Formatted at 2026-08-12 02:36:08 on dist-test-slave-mn4r"
server_key: "c3773998d981356b2fa4c71ec8b9539c"
server_key_iv: "fbeb5291545b5468b1462446f001d28b"
server_key_version: "encryptionkey@0"
I20260812 02:36:08.580829 9187 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502156971023-7698-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502156971023-7698-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502156971023-7698-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:36:08.597958 9187 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:36:08.602061 9187 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:36:08.602394 9187 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:36:08.605283 9187 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:36:08.605404 9187 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:36:08.605563 9187 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:36:08.605643 9187 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 02:36:08 dist-test-slave-mn4r krb5kdc[9040](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502168, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.129@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 02:36:08.629726 9215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:08.605881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:56183 (local address 127.7.132.190:43261)
0812 02:36:08.606627 (+ 746us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:08.606633 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:08.607750 (+ 1117us) server_negotiation.cc:407] Connection header received
0812 02:36:08.609889 (+ 2139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:08.609894 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:08.610012 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:08.610160 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:08.612493 (+ 2333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:08.614005 (+ 1512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:08.616792 (+ 2787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:08.617430 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:08.617579 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:08.622649 (+ 5070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:08.622654 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:08.622658 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:08.622693 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:08.625391 (+ 2698us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:08.626611 (+ 1220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:08.626618 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:08.626621 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:08.626792 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:08.627460 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:08.627468 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:08.627471 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:08.627911 (+ 440us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:08.628082 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:08.629254 (+ 1172us) server_negotiation.cc:299] Negotiation successful
0812 02:36:08.629446 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":610,"thread_start_us":489,"threads_started":1}
I20260812 02:36:08.632061 9216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:08.606626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:43261 (local address 127.7.132.129:56183)
0812 02:36:08.607551 (+ 925us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:08.607602 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:08.609602 (+ 2000us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:08.610439 (+ 837us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:08.610453 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:08.611362 (+ 909us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:36:08.612221 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:08.612277 (+ 56us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:08.614352 (+ 2075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:08.614360 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:08.616631 (+ 2271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:08.616645 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:08.617719 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:08.617727 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:08.618060 (+ 333us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:08.618926 (+ 866us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:36:08.618967 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:36:08.622313 (+ 3346us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:36:08.625565 (+ 3252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:08.625579 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:08.625603 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:08.626435 (+ 832us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:08.626943 (+ 508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:08.626948 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:08.626952 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:08.627296 (+ 344us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:08.628077 (+ 781us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:36:08.628090 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:36:08.628762 (+ 672us) client_negotiation.cc:769] Sending connection context
0812 02:36:08.629177 (+ 415us) client_negotiation.cc:240] Negotiation successful
0812 02:36:08.629961 (+ 784us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":622,"thread_start_us":375,"threads_started":1}
I20260812 02:36:08.657810 9215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:08.639239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:52947 (local address 127.7.132.190:43261)
0812 02:36:08.639398 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:08.639402 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:08.640198 (+ 796us) server_negotiation.cc:407] Connection header received
0812 02:36:08.640490 (+ 292us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:08.640496 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:08.640579 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:08.640695 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:08.642490 (+ 1795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:08.643639 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:08.646281 (+ 2642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:08.646875 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:08.646999 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:08.650186 (+ 3187us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:08.650197 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:08.650201 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:08.650238 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:08.654421 (+ 4183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:08.655365 (+ 944us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:08.655372 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:08.655375 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:08.655573 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:08.656112 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:08.656120 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:08.656124 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:08.656458 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:08.656648 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:08.657124 (+ 476us) server_negotiation.cc:299] Negotiation successful
0812 02:36:08.657379 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 02:36:08.657938 9295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:08.639348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:43261 (local address 127.7.132.129:52947)
0812 02:36:08.640059 (+ 711us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:08.640077 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:08.640296 (+ 219us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:08.640740 (+ 444us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:08.640749 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:08.641647 (+ 898us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:36:08.642231 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:08.642256 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:08.643947 (+ 1691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:08.643951 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:08.646020 (+ 2069us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:08.646038 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:08.647405 (+ 1367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:08.647416 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:08.647671 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:08.648537 (+ 866us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:36:08.648571 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:36:08.649879 (+ 1308us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:36:08.654588 (+ 4709us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:08.654596 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:08.654603 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:08.655204 (+ 601us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:08.655737 (+ 533us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:08.655748 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:08.655754 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:08.655963 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:08.656610 (+ 647us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:36:08.656617 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:36:08.656830 (+ 213us) client_negotiation.cc:769] Sending connection context
0812 02:36:08.657001 (+ 171us) client_negotiation.cc:240] Negotiation successful
0812 02:36:08.657380 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":568,"thread_start_us":391,"threads_started":1}
I20260812 02:36:08.660777 9187 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.129:34685
I20260812 02:36:08.660801 9348 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.129:34685 every 8 connection(s)
I20260812 02:36:08.662561 9187 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502156971023-7698-0/minicluster-data/ts-0/data/info.pb
I20260812 02:36:08.664582 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 9187
I20260812 02:36:08.664806 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502156971023-7698-0/minicluster-data/ts-0/wal/instance
I20260812 02:36:08.665119 9349 heartbeater.cc:347] Connected to a master server at 127.7.132.190:43261
I20260812 02:36:08.665359 9349 heartbeater.cc:464] Registering TS with master...
I20260812 02:36:08.665606 7698 external_mini_cluster.cc:1428] Setting key e95d13b2f3ab1f41058eed34e29379b6
I20260812 02:36:08.666049 9349 heartbeater.cc:511] Master 127.7.132.190:43261 requested a full tablet report, sending...
I20260812 02:36:08.668511 9096 ts_manager.cc:194] Registered new tserver with Master: 3c7b0dc3dde34ef98a501ea15c294a1b (127.7.132.129:34685)
I20260812 02:36:08.670817 9096 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.129@KRBTEST.COM'} at 127.7.132.129:56183
WARNING: no policy specified for kudu/127.7.132.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.130.keytab.
Entry for principal kudu/127.7.132.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.130.keytab.
I20260812 02:36:08.713933 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502156971023-7698-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502156971023-7698-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502156971023-7698-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502156971023-7698-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.130:0
--local_ip_for_outbound_sockets=127.7.132.130
--webserver_interface=127.7.132.130
--webserver_port=0
--tserver_master_addrs=127.7.132.190:43261
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.130.keytab
--principal=kudu/127.7.132.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:43895
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:36:09.051481 9355 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:36:09.051810 9355 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:36:09.051991 9355 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:36:09.059230 9355 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:36:09.059338 9355 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:36:09.059486 9355 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 02:36:09.059578 9355 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.130
I20260812 02:36:09.069693 9355 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:43895
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502156971023-7698-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502156971023-7698-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.130.keytab
--principal=kudu/127.7.132.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502156971023-7698-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.132.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.132.190:43261
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.130
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502156971023-7698-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:36:09.071100 9355 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:36:09.072484 9355 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:36:09.086638 9361 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:36:09.086570 9363 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:36:09.086570 9360 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:36:09.088346 9355 server_base.cc:1034] running on GCE node
I20260812 02:36:09.089022 9355 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:36:09.090023 9355 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:36:09.091290 9355 hybrid_clock.cc:648] HybridClock initialized: now 1786502169091215 us; error 59 us; skew 500 ppm
I20260812 02:36:09.091611 9355 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:36:09 dist-test-slave-mn4r krb5kdc[9040](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502169, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:36:09.095327 9355 init.cc:377] Logged in from keytab as kudu/127.7.132.130@KRBTEST.COM (short username kudu)
I20260812 02:36:09.096845 9355 webserver.cc:460] Webserver started at http://127.7.132.130:41093/ using document root <none> and password file <none>
I20260812 02:36:09.097641 9355 fs_manager.cc:359] Metadata directory not provided
I20260812 02:36:09.097756 9355 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:36:09.098163 9355 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:36:09.101161 9355 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502156971023-7698-0/minicluster-data/ts-1/data/instance:
uuid: "be444de7c1c94ee8a92f2a78f03e654f"
format_stamp: "Formatted at 2026-08-12 02:36:09 on dist-test-slave-mn4r"
server_key: "05be082aa5a383d0a8d349c2eebe9ad3"
server_key_iv: "9a9034676bebc6012ae7035acdd9382a"
server_key_version: "encryptionkey@0"
I20260812 02:36:09.101987 9355 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502156971023-7698-0/minicluster-data/ts-1/wal/instance:
uuid: "be444de7c1c94ee8a92f2a78f03e654f"
format_stamp: "Formatted at 2026-08-12 02:36:09 on dist-test-slave-mn4r"
server_key: "05be082aa5a383d0a8d349c2eebe9ad3"
server_key_iv: "9a9034676bebc6012ae7035acdd9382a"
server_key_version: "encryptionkey@0"
I20260812 02:36:09.107447 9355 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260812 02:36:09.111157 9372 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:36:09.112583 9355 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 02:36:09.112778 9355 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502156971023-7698-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502156971023-7698-0/minicluster-data/ts-1/wal
uuid: "be444de7c1c94ee8a92f2a78f03e654f"
format_stamp: "Formatted at 2026-08-12 02:36:09 on dist-test-slave-mn4r"
server_key: "05be082aa5a383d0a8d349c2eebe9ad3"
server_key_iv: "9a9034676bebc6012ae7035acdd9382a"
server_key_version: "encryptionkey@0"
I20260812 02:36:09.113003 9355 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502156971023-7698-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502156971023-7698-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502156971023-7698-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:36:09.145336 9355 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:36:09.149930 9355 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:36:09.150244 9355 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:36:09.153317 9355 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:36:09.153434 9355 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:36:09.153585 9355 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:36:09.153676 9355 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 02:36:09 dist-test-slave-mn4r krb5kdc[9040](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502169, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.130@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 02:36:09.180771 9382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:09.154490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:43261 (local address 127.7.132.130:52885)
0812 02:36:09.155465 (+ 975us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:09.155519 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:09.158418 (+ 2899us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:09.159352 (+ 934us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:09.159364 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:09.160491 (+ 1127us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:36:09.161706 (+ 1215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:09.161745 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:09.163201 (+ 1456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:09.163205 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:09.165668 (+ 2463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:09.165680 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:09.166781 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:09.166790 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:09.167036 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:09.167736 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:36:09.167758 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:36:09.171179 (+ 3421us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:36:09.174916 (+ 3737us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:09.174924 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:09.174951 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:09.175708 (+ 757us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:09.176217 (+ 509us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:09.176224 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:09.176228 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:09.176536 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:09.177284 (+ 748us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:36:09.177292 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:36:09.177863 (+ 571us) client_negotiation.cc:769] Sending connection context
0812 02:36:09.178305 (+ 442us) client_negotiation.cc:240] Negotiation successful
0812 02:36:09.178935 (+ 630us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":673,"thread_start_us":362,"threads_started":1}
I20260812 02:36:09.181830 9215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:09.153688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:52885 (local address 127.7.132.190:43261)
0812 02:36:09.153926 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:09.153934 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:09.155658 (+ 1724us) server_negotiation.cc:407] Connection header received
0812 02:36:09.158794 (+ 3136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:09.158800 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:09.158893 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:09.159019 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:09.161979 (+ 2960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:09.163012 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:09.165837 (+ 2825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:09.166523 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:09.166666 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:09.171476 (+ 4810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:09.171482 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:09.171488 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:09.171536 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:09.174749 (+ 3213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:09.175874 (+ 1125us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:09.175880 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:09.175885 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:09.176052 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:09.176667 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:09.176675 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:09.176680 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:09.177131 (+ 451us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:09.177286 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:09.180946 (+ 3660us) server_negotiation.cc:299] Negotiation successful
0812 02:36:09.181360 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260812 02:36:09.205125 9215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:09.188201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:41951 (local address 127.7.132.190:43261)
0812 02:36:09.188424 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:09.188431 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:09.189096 (+ 665us) server_negotiation.cc:407] Connection header received
0812 02:36:09.189289 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:09.189296 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:09.189386 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:09.189495 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:09.191020 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:09.191724 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:09.194324 (+ 2600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:09.194906 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:09.195071 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:09.198284 (+ 3213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:09.198291 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:09.198296 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:09.198341 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:09.201851 (+ 3510us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:09.202865 (+ 1014us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:09.202870 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:09.202874 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:09.203011 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:09.203401 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:09.203409 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:09.203414 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:09.203867 (+ 453us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:09.204054 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:09.204541 (+ 487us) server_negotiation.cc:299] Negotiation successful
0812 02:36:09.204762 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260812 02:36:09.205240 9466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:09.188237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:43261 (local address 127.7.132.130:41951)
0812 02:36:09.188985 (+ 748us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:09.189007 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:09.189165 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:09.189561 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:09.189573 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:09.190350 (+ 777us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:36:09.190831 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:09.190849 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:09.191880 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:09.191888 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:09.194149 (+ 2261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:09.194172 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:09.195187 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:09.195198 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:09.195466 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:09.196431 (+ 965us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:36:09.196468 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:36:09.198032 (+ 1564us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:36:09.202010 (+ 3978us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:09.202016 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:09.202021 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:09.202671 (+ 650us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:09.203138 (+ 467us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:09.203143 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:09.203146 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:09.203280 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:09.204120 (+ 840us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:36:09.204124 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:36:09.204285 (+ 161us) client_negotiation.cc:769] Sending connection context
0812 02:36:09.204429 (+ 144us) client_negotiation.cc:240] Negotiation successful
0812 02:36:09.204781 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":626,"thread_start_us":470,"threads_started":1}
I20260812 02:36:09.215629 9355 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.130:42511
I20260812 02:36:09.215652 9519 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.130:42511 every 8 connection(s)
I20260812 02:36:09.216974 9355 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502156971023-7698-0/minicluster-data/ts-1/data/info.pb
I20260812 02:36:09.219058 9520 heartbeater.cc:347] Connected to a master server at 127.7.132.190:43261
I20260812 02:36:09.219280 9520 heartbeater.cc:464] Registering TS with master...
I20260812 02:36:09.219928 9520 heartbeater.cc:511] Master 127.7.132.190:43261 requested a full tablet report, sending...
I20260812 02:36:09.221457 9096 ts_manager.cc:194] Registered new tserver with Master: be444de7c1c94ee8a92f2a78f03e654f (127.7.132.130:42511)
I20260812 02:36:09.222406 9096 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.130@KRBTEST.COM'} at 127.7.132.130:52885
I20260812 02:36:09.224767 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 9355
I20260812 02:36:09.224973 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502156971023-7698-0/minicluster-data/ts-1/wal/instance
I20260812 02:36:09.225387 7698 external_mini_cluster.cc:1428] Setting key 2f9422008f89a9fa82f963e8c494b0f9
WARNING: no policy specified for kudu/127.7.132.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.131.keytab.
Entry for principal kudu/127.7.132.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.131.keytab.
I20260812 02:36:09.273536 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502156971023-7698-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502156971023-7698-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502156971023-7698-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502156971023-7698-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.131:0
--local_ip_for_outbound_sockets=127.7.132.131
--webserver_interface=127.7.132.131
--webserver_port=0
--tserver_master_addrs=127.7.132.190:43261
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.131.keytab
--principal=kudu/127.7.132.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:43895
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:36:09.618407 9525 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:36:09.618695 9525 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:36:09.618885 9525 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:36:09.626314 9525 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:36:09.626449 9525 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:36:09.626607 9525 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 02:36:09.626739 9525 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.131
I20260812 02:36:09.637696 9525 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:43895
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502156971023-7698-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502156971023-7698-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.131.keytab
--principal=kudu/127.7.132.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502156971023-7698-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.132.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.132.190:43261
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.131
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502156971023-7698-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:36:09.639128 9525 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:36:09.640560 9525 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:36:09.654747 9531 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:36:09.654717 9534 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:36:09.654808 9532 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:36:09.655812 9525 server_base.cc:1034] running on GCE node
I20260812 02:36:09.656476 9525 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:36:09.657465 9525 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:36:09.658723 9525 hybrid_clock.cc:648] HybridClock initialized: now 1786502169658652 us; error 42 us; skew 500 ppm
I20260812 02:36:09.659042 9525 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:36:09 dist-test-slave-mn4r krb5kdc[9040](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502169, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:36:09.663200 9525 init.cc:377] Logged in from keytab as kudu/127.7.132.131@KRBTEST.COM (short username kudu)
I20260812 02:36:09.665056 9525 webserver.cc:460] Webserver started at http://127.7.132.131:45719/ using document root <none> and password file <none>
I20260812 02:36:09.665820 9525 fs_manager.cc:359] Metadata directory not provided
I20260812 02:36:09.665958 9525 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:36:09.666342 9525 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:36:09.669239 9525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502156971023-7698-0/minicluster-data/ts-2/data/instance:
uuid: "ea68384054b64897b6c13998b543bd1b"
format_stamp: "Formatted at 2026-08-12 02:36:09 on dist-test-slave-mn4r"
server_key: "05cda8c6aa7138031e5ab179bd6e5da6"
server_key_iv: "6d84c27b01cd633c1e1ce57352737edf"
server_key_version: "encryptionkey@0"
I20260812 02:36:09.670158 9525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502156971023-7698-0/minicluster-data/ts-2/wal/instance:
uuid: "ea68384054b64897b6c13998b543bd1b"
format_stamp: "Formatted at 2026-08-12 02:36:09 on dist-test-slave-mn4r"
server_key: "05cda8c6aa7138031e5ab179bd6e5da6"
server_key_iv: "6d84c27b01cd633c1e1ce57352737edf"
server_key_version: "encryptionkey@0"
I20260812 02:36:09.673735 9349 heartbeater.cc:503] Master 127.7.132.190:43261 was elected leader, sending a full tablet report...
I20260812 02:36:09.675655 9525 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260812 02:36:09.679347 9541 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:36:09.680775 9525 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260812 02:36:09.680971 9525 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502156971023-7698-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502156971023-7698-0/minicluster-data/ts-2/wal
uuid: "ea68384054b64897b6c13998b543bd1b"
format_stamp: "Formatted at 2026-08-12 02:36:09 on dist-test-slave-mn4r"
server_key: "05cda8c6aa7138031e5ab179bd6e5da6"
server_key_iv: "6d84c27b01cd633c1e1ce57352737edf"
server_key_version: "encryptionkey@0"
I20260812 02:36:09.681238 9525 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502156971023-7698-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502156971023-7698-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502156971023-7698-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:36:09.699503 9525 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:36:09.704382 9525 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:36:09.704694 9525 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:36:09.707801 9525 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:36:09.707914 9525 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:36:09.708076 9525 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:36:09.708199 9525 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
Aug 12 02:36:09 dist-test-slave-mn4r krb5kdc[9040](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502169, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.131@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 02:36:09.735563 9554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:09.708340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:49677 (local address 127.7.132.190:43261)
0812 02:36:09.709211 (+ 871us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:09.709217 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:09.710106 (+ 889us) server_negotiation.cc:407] Connection header received
0812 02:36:09.712333 (+ 2227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:09.712339 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:09.712437 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:09.712573 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:09.715007 (+ 2434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:09.716161 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:09.719391 (+ 3230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:09.720222 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:09.720368 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:09.726166 (+ 5798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:09.726175 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:09.726182 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:09.726242 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:09.730842 (+ 4600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:09.732144 (+ 1302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:09.732152 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:09.732157 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:09.732343 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:09.733052 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:09.733058 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:09.733061 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:09.733367 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:09.733480 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:09.734761 (+ 1281us) server_negotiation.cc:299] Negotiation successful
0812 02:36:09.735069 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":763,"thread_start_us":668,"threads_started":1}
I20260812 02:36:09.737848 9555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:09.709018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:43261 (local address 127.7.132.131:49677)
0812 02:36:09.709890 (+ 872us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:09.709941 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:09.712038 (+ 2097us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:09.712846 (+ 808us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:09.712854 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:09.713727 (+ 873us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:36:09.714807 (+ 1080us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:09.714834 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:09.716387 (+ 1553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:09.716395 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:09.719183 (+ 2788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:09.719206 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:09.720474 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:09.720479 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:09.720721 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:09.721642 (+ 921us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:36:09.721691 (+ 49us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:36:09.725760 (+ 4069us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:36:09.731023 (+ 5263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:09.731036 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:09.731066 (+ 30us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:09.731974 (+ 908us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:09.732490 (+ 516us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:09.732498 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:09.732516 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:09.732855 (+ 339us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:09.733518 (+ 663us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:36:09.733534 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:36:09.734171 (+ 637us) client_negotiation.cc:769] Sending connection context
0812 02:36:09.734631 (+ 460us) client_negotiation.cc:240] Negotiation successful
0812 02:36:09.735387 (+ 756us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":622,"thread_start_us":403,"threads_started":1}
I20260812 02:36:09.753811 9525 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.131:41005
I20260812 02:36:09.753849 9695 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.131:41005 every 8 connection(s)
I20260812 02:36:09.755180 9525 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502156971023-7698-0/minicluster-data/ts-2/data/info.pb
I20260812 02:36:09.757612 9697 heartbeater.cc:347] Connected to a master server at 127.7.132.190:43261
I20260812 02:36:09.757858 9697 heartbeater.cc:464] Registering TS with master...
I20260812 02:36:09.758592 9697 heartbeater.cc:511] Master 127.7.132.190:43261 requested a full tablet report, sending...
I20260812 02:36:09.760017 9096 ts_manager.cc:194] Registered new tserver with Master: ea68384054b64897b6c13998b543bd1b (127.7.132.131:41005)
I20260812 02:36:09.760118 9554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:09.744803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:42493 (local address 127.7.132.190:43261)
0812 02:36:09.744980 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:09.744983 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:09.745718 (+ 735us) server_negotiation.cc:407] Connection header received
0812 02:36:09.745917 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:09.745922 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:09.746015 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:09.746155 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:09.747766 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:09.748719 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:09.750863 (+ 2144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:09.751623 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:09.751767 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:09.753865 (+ 2098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:09.753871 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:09.753875 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:09.753919 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:09.757135 (+ 3216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:09.758041 (+ 906us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:09.758045 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:09.758048 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:09.758230 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:09.758613 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:09.758619 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:09.758623 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:09.758910 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:09.759088 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:09.759478 (+ 390us) server_negotiation.cc:299] Negotiation successful
0812 02:36:09.759727 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 02:36:09.760193 9666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:09.744978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:43261 (local address 127.7.132.131:42493)
0812 02:36:09.745613 (+ 635us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:09.745631 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:09.745792 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:09.746214 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:09.746219 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:09.747073 (+ 854us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:36:09.747573 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:09.747590 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:09.748908 (+ 1318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:09.748919 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:09.750709 (+ 1790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:09.750725 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:09.751807 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:09.751813 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:09.751992 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:09.752653 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:36:09.752670 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:36:09.753673 (+ 1003us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:36:09.757285 (+ 3612us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:09.757292 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:09.757295 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:09.757868 (+ 573us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:09.758346 (+ 478us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:09.758354 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:09.758358 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:09.758493 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:09.759053 (+ 560us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:36:09.759058 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:36:09.759238 (+ 180us) client_negotiation.cc:769] Sending connection context
0812 02:36:09.759382 (+ 144us) client_negotiation.cc:240] Negotiation successful
0812 02:36:09.759715 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":514,"thread_start_us":370,"threads_started":1}
I20260812 02:36:09.761094 9096 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.131@KRBTEST.COM'} at 127.7.132.131:49677
I20260812 02:36:09.765333 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 9525
I20260812 02:36:09.765565 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502156971023-7698-0/minicluster-data/ts-2/wal/instance
I20260812 02:36:09.766006 7698 external_mini_cluster.cc:1428] Setting key 2fe782ec805b122934709b539744778c
I20260812 02:36:09.769044 7698 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 02:36:09 dist-test-slave-mn4r krb5kdc[9040](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502169, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 02:36:09 dist-test-slave-mn4r krb5kdc[9040](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502169, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 02:36:09.820242 9554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:09.805881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50278 (local address 127.7.132.190:43261)
0812 02:36:09.806196 (+ 315us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:09.806205 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:09.806412 (+ 207us) server_negotiation.cc:407] Connection header received
0812 02:36:09.806579 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:09.806584 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:09.806656 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:09.806753 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:09.808099 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:09.808936 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:09.810238 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:09.810692 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:09.810798 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:09.814709 (+ 3911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:09.814714 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:09.814718 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:09.814754 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:09.817698 (+ 2944us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:09.818405 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:09.818410 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:09.818413 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:09.818562 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:09.818892 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:09.818896 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:09.818898 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:09.819177 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:09.819380 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:09.819644 (+ 264us) server_negotiation.cc:299] Negotiation successful
0812 02:36:09.819817 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260812 02:36:09.824405 9095 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50278:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 02:36:09.831444 9095 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 02:36:09.869700 9721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:09.845421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55008 (local address 127.7.132.129:34685)
0812 02:36:09.846053 (+ 632us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:09.846064 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:09.857973 (+ 11909us) server_negotiation.cc:407] Connection header received
0812 02:36:09.859531 (+ 1558us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:09.859546 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:09.859935 (+ 389us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:09.860167 (+ 232us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:36:09.861558 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:09.863616 (+ 2058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:09.867492 (+ 3876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:09.868426 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:09.868599 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:09.868848 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:09.869194 (+ 346us) server_negotiation.cc:299] Negotiation successful
0812 02:36:09.869385 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":463,"thread_start_us":313,"threads_started":1}
I20260812 02:36:09.869969 9723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:09.845893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49636 (local address 127.7.132.130:42511)
0812 02:36:09.856851 (+ 10958us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:09.856885 (+ 34us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:09.860499 (+ 3614us) server_negotiation.cc:407] Connection header received
0812 02:36:09.861135 (+ 636us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:09.861147 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:09.861391 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:09.861573 (+ 182us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:36:09.862618 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:09.863724 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:09.867089 (+ 3365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:09.868568 (+ 1479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:09.868678 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:09.868881 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:09.869509 (+ 628us) server_negotiation.cc:299] Negotiation successful
0812 02:36:09.869681 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10701,"thread_start_us":10539,"threads_started":1}
I20260812 02:36:09.870810 9724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:09.845236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:34685 (local address 127.0.0.1:55008)
0812 02:36:09.857698 (+ 12462us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:09.857768 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:09.859165 (+ 1397us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:09.860612 (+ 1447us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:09.860631 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:09.860675 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:36:09.861287 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:09.861306 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:09.863847 (+ 2541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:09.863854 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:09.867326 (+ 3472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:09.867338 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:09.868659 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:09.868666 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:09.868861 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:09.868921 (+ 60us) client_negotiation.cc:769] Sending connection context
0812 02:36:09.870311 (+ 1390us) client_negotiation.cc:240] Negotiation successful
0812 02:36:09.870463 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12086,"mutex_wait_us":94,"thread_start_us":15297,"threads_started":1}
I20260812 02:36:09.871122 9719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:09.845421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.130:42511 (local address 127.0.0.1:49636)
0812 02:36:09.860340 (+ 14919us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:09.860365 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:09.860846 (+ 481us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:09.861828 (+ 982us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:09.861831 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:09.861860 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:36:09.862426 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:09.862439 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:09.863994 (+ 1555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:09.863998 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:09.866825 (+ 2827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:09.866837 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:09.869099 (+ 2262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:09.869107 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:09.869290 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:09.869332 (+ 42us) client_negotiation.cc:769] Sending connection context
0812 02:36:09.870464 (+ 1132us) client_negotiation.cc:240] Negotiation successful
0812 02:36:09.870607 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14744,"mutex_wait_us":71,"thread_start_us":16362,"threads_started":1}
I20260812 02:36:09.871621 9722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:09.846593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.131:41005 (local address 127.0.0.1:54050)
0812 02:36:09.861595 (+ 15002us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:09.861614 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:09.861715 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:09.864018 (+ 2303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:09.864023 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:09.864055 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:36:09.864639 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:09.864653 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:09.866963 (+ 2310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:09.866968 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:09.869029 (+ 2061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:09.869041 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:09.870919 (+ 1878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:09.870928 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:09.871128 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:09.871161 (+ 33us) client_negotiation.cc:769] Sending connection context
0812 02:36:09.871268 (+ 107us) client_negotiation.cc:240] Negotiation successful
0812 02:36:09.871399 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14864,"thread_start_us":11419,"threads_started":1}
I20260812 02:36:09.872340 9725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:09.847042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54050 (local address 127.7.132.131:41005)
0812 02:36:09.863411 (+ 16369us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:09.863421 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:09.863453 (+ 32us) server_negotiation.cc:407] Connection header received
0812 02:36:09.863551 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:09.863558 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:09.863807 (+ 249us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:09.864045 (+ 238us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:36:09.864928 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:09.866815 (+ 1887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:09.869212 (+ 2397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:09.870694 (+ 1482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:09.870832 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:09.871057 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:09.871871 (+ 814us) server_negotiation.cc:299] Negotiation successful
0812 02:36:09.872040 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16171,"thread_start_us":17274,"threads_started":1}
I20260812 02:36:09.875238 9429 tablet_service.cc:1467] Processing CreateTablet for tablet 9040dea9bdfb49998886c2d047d46601 (DEFAULT_TABLE table=test-table [id=a79546a4eed84eb89aa88c0b6f0de479]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:36:09.875967 9608 tablet_service.cc:1467] Processing CreateTablet for tablet 9040dea9bdfb49998886c2d047d46601 (DEFAULT_TABLE table=test-table [id=a79546a4eed84eb89aa88c0b6f0de479]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:36:09.876812 9429 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9040dea9bdfb49998886c2d047d46601. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:36:09.877466 9608 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9040dea9bdfb49998886c2d047d46601. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:36:09.880424 9264 tablet_service.cc:1467] Processing CreateTablet for tablet 9040dea9bdfb49998886c2d047d46601 (DEFAULT_TABLE table=test-table [id=a79546a4eed84eb89aa88c0b6f0de479]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:36:09.882196 9264 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9040dea9bdfb49998886c2d047d46601. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:36:09.892351 9727 tablet_bootstrap.cc:492] T 9040dea9bdfb49998886c2d047d46601 P ea68384054b64897b6c13998b543bd1b: Bootstrap starting.
I20260812 02:36:09.893581 9726 tablet_bootstrap.cc:492] T 9040dea9bdfb49998886c2d047d46601 P be444de7c1c94ee8a92f2a78f03e654f: Bootstrap starting.
I20260812 02:36:09.895596 9728 tablet_bootstrap.cc:492] T 9040dea9bdfb49998886c2d047d46601 P 3c7b0dc3dde34ef98a501ea15c294a1b: Bootstrap starting.
I20260812 02:36:09.897982 9726 tablet_bootstrap.cc:654] T 9040dea9bdfb49998886c2d047d46601 P be444de7c1c94ee8a92f2a78f03e654f: Neither blocks nor log segments found. Creating new log.
I20260812 02:36:09.898828 9728 tablet_bootstrap.cc:654] T 9040dea9bdfb49998886c2d047d46601 P 3c7b0dc3dde34ef98a501ea15c294a1b: Neither blocks nor log segments found. Creating new log.
I20260812 02:36:09.899535 9726 log.cc:824] T 9040dea9bdfb49998886c2d047d46601 P be444de7c1c94ee8a92f2a78f03e654f: Log is configured to *not* fsync() on all Append() calls
I20260812 02:36:09.899883 9728 log.cc:824] T 9040dea9bdfb49998886c2d047d46601 P 3c7b0dc3dde34ef98a501ea15c294a1b: Log is configured to *not* fsync() on all Append() calls
I20260812 02:36:09.899955 9727 tablet_bootstrap.cc:654] T 9040dea9bdfb49998886c2d047d46601 P ea68384054b64897b6c13998b543bd1b: Neither blocks nor log segments found. Creating new log.
I20260812 02:36:09.901345 9727 log.cc:824] T 9040dea9bdfb49998886c2d047d46601 P ea68384054b64897b6c13998b543bd1b: Log is configured to *not* fsync() on all Append() calls
I20260812 02:36:09.903451 9728 tablet_bootstrap.cc:492] T 9040dea9bdfb49998886c2d047d46601 P 3c7b0dc3dde34ef98a501ea15c294a1b: No bootstrap required, opened a new log
I20260812 02:36:09.903776 9726 tablet_bootstrap.cc:492] T 9040dea9bdfb49998886c2d047d46601 P be444de7c1c94ee8a92f2a78f03e654f: No bootstrap required, opened a new log
I20260812 02:36:09.903786 9728 ts_tablet_manager.cc:1397] T 9040dea9bdfb49998886c2d047d46601 P 3c7b0dc3dde34ef98a501ea15c294a1b: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.007s
I20260812 02:36:09.904057 9726 ts_tablet_manager.cc:1397] T 9040dea9bdfb49998886c2d047d46601 P be444de7c1c94ee8a92f2a78f03e654f: Time spent bootstrapping tablet: real 0.011s user 0.009s sys 0.000s
I20260812 02:36:09.905092 9727 tablet_bootstrap.cc:492] T 9040dea9bdfb49998886c2d047d46601 P ea68384054b64897b6c13998b543bd1b: No bootstrap required, opened a new log
I20260812 02:36:09.905382 9727 ts_tablet_manager.cc:1397] T 9040dea9bdfb49998886c2d047d46601 P ea68384054b64897b6c13998b543bd1b: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.004s
I20260812 02:36:09.911629 9726 raft_consensus.cc:348] T 9040dea9bdfb49998886c2d047d46601 P be444de7c1c94ee8a92f2a78f03e654f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c7b0dc3dde34ef98a501ea15c294a1b" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 34685 } } peers { permanent_uuid: "be444de7c1c94ee8a92f2a78f03e654f" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 42511 } } peers { permanent_uuid: "ea68384054b64897b6c13998b543bd1b" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 41005 } }
I20260812 02:36:09.912041 9726 raft_consensus.cc:374] T 9040dea9bdfb49998886c2d047d46601 P be444de7c1c94ee8a92f2a78f03e654f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:36:09.912155 9726 raft_consensus.cc:729] T 9040dea9bdfb49998886c2d047d46601 P be444de7c1c94ee8a92f2a78f03e654f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be444de7c1c94ee8a92f2a78f03e654f, State: Initialized, Role: FOLLOWER
I20260812 02:36:09.912834 9726 consensus_queue.cc:260] T 9040dea9bdfb49998886c2d047d46601 P be444de7c1c94ee8a92f2a78f03e654f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c7b0dc3dde34ef98a501ea15c294a1b" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 34685 } } peers { permanent_uuid: "be444de7c1c94ee8a92f2a78f03e654f" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 42511 } } peers { permanent_uuid: "ea68384054b64897b6c13998b543bd1b" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 41005 } }
I20260812 02:36:09.914321 9520 heartbeater.cc:503] Master 127.7.132.190:43261 was elected leader, sending a full tablet report...
I20260812 02:36:09.914896 9726 ts_tablet_manager.cc:1428] T 9040dea9bdfb49998886c2d047d46601 P be444de7c1c94ee8a92f2a78f03e654f: Time spent starting tablet: real 0.011s user 0.011s sys 0.000s
I20260812 02:36:09.915998 9728 raft_consensus.cc:348] T 9040dea9bdfb49998886c2d047d46601 P 3c7b0dc3dde34ef98a501ea15c294a1b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c7b0dc3dde34ef98a501ea15c294a1b" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 34685 } } peers { permanent_uuid: "be444de7c1c94ee8a92f2a78f03e654f" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 42511 } } peers { permanent_uuid: "ea68384054b64897b6c13998b543bd1b" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 41005 } }
I20260812 02:36:09.916507 9728 raft_consensus.cc:374] T 9040dea9bdfb49998886c2d047d46601 P 3c7b0dc3dde34ef98a501ea15c294a1b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:36:09.916684 9728 raft_consensus.cc:729] T 9040dea9bdfb49998886c2d047d46601 P 3c7b0dc3dde34ef98a501ea15c294a1b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c7b0dc3dde34ef98a501ea15c294a1b, State: Initialized, Role: FOLLOWER
I20260812 02:36:09.917491 9728 consensus_queue.cc:260] T 9040dea9bdfb49998886c2d047d46601 P 3c7b0dc3dde34ef98a501ea15c294a1b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c7b0dc3dde34ef98a501ea15c294a1b" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 34685 } } peers { permanent_uuid: "be444de7c1c94ee8a92f2a78f03e654f" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 42511 } } peers { permanent_uuid: "ea68384054b64897b6c13998b543bd1b" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 41005 } }
I20260812 02:36:09.919441 9728 ts_tablet_manager.cc:1428] T 9040dea9bdfb49998886c2d047d46601 P 3c7b0dc3dde34ef98a501ea15c294a1b: Time spent starting tablet: real 0.016s user 0.014s sys 0.002s
I20260812 02:36:09.921077 9727 raft_consensus.cc:348] T 9040dea9bdfb49998886c2d047d46601 P ea68384054b64897b6c13998b543bd1b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c7b0dc3dde34ef98a501ea15c294a1b" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 34685 } } peers { permanent_uuid: "be444de7c1c94ee8a92f2a78f03e654f" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 42511 } } peers { permanent_uuid: "ea68384054b64897b6c13998b543bd1b" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 41005 } }
I20260812 02:36:09.921504 9727 raft_consensus.cc:374] T 9040dea9bdfb49998886c2d047d46601 P ea68384054b64897b6c13998b543bd1b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:36:09.921620 9727 raft_consensus.cc:729] T 9040dea9bdfb49998886c2d047d46601 P ea68384054b64897b6c13998b543bd1b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea68384054b64897b6c13998b543bd1b, State: Initialized, Role: FOLLOWER
I20260812 02:36:09.922312 9727 consensus_queue.cc:260] T 9040dea9bdfb49998886c2d047d46601 P ea68384054b64897b6c13998b543bd1b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c7b0dc3dde34ef98a501ea15c294a1b" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 34685 } } peers { permanent_uuid: "be444de7c1c94ee8a92f2a78f03e654f" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 42511 } } peers { permanent_uuid: "ea68384054b64897b6c13998b543bd1b" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 41005 } }
I20260812 02:36:09.923484 9697 heartbeater.cc:503] Master 127.7.132.190:43261 was elected leader, sending a full tablet report...
I20260812 02:36:09.924239 9727 ts_tablet_manager.cc:1428] T 9040dea9bdfb49998886c2d047d46601 P ea68384054b64897b6c13998b543bd1b: Time spent starting tablet: real 0.019s user 0.016s sys 0.000s
W20260812 02:36:09.969089 9521 tablet.cc:2369] T 9040dea9bdfb49998886c2d047d46601 P be444de7c1c94ee8a92f2a78f03e654f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 02:36:10.006970 9700 tablet.cc:2369] T 9040dea9bdfb49998886c2d047d46601 P ea68384054b64897b6c13998b543bd1b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 02:36:10.026661 9733 raft_consensus.cc:482] T 9040dea9bdfb49998886c2d047d46601 P be444de7c1c94ee8a92f2a78f03e654f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 02:36:10.026937 9733 raft_consensus.cc:504] T 9040dea9bdfb49998886c2d047d46601 P be444de7c1c94ee8a92f2a78f03e654f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c7b0dc3dde34ef98a501ea15c294a1b" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 34685 } } peers { permanent_uuid: "be444de7c1c94ee8a92f2a78f03e654f" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 42511 } } peers { permanent_uuid: "ea68384054b64897b6c13998b543bd1b" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 41005 } }
I20260812 02:36:10.028654 9733 leader_election.cc:288] T 9040dea9bdfb49998886c2d047d46601 P be444de7c1c94ee8a92f2a78f03e654f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c7b0dc3dde34ef98a501ea15c294a1b (127.7.132.129:34685), ea68384054b64897b6c13998b543bd1b (127.7.132.131:41005)
I20260812 02:36:10.036180 9721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:10.029016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:46509 (local address 127.7.132.129:34685)
0812 02:36:10.029248 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:10.029254 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:10.030286 (+ 1032us) server_negotiation.cc:407] Connection header received
0812 02:36:10.030468 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:10.030474 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:10.030563 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:10.030660 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:36:10.031362 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:10.032152 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:10.034046 (+ 1894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:10.035009 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:10.035139 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:10.035193 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:10.035799 (+ 606us) server_negotiation.cc:299] Negotiation successful
0812 02:36:10.035931 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260812 02:36:10.036229 9725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:10.029299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:42143 (local address 127.7.132.131:41005)
0812 02:36:10.029539 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:10.029548 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:10.030511 (+ 963us) server_negotiation.cc:407] Connection header received
0812 02:36:10.030696 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:10.030701 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:10.030778 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:10.030882 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:36:10.031537 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:10.032294 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:10.034165 (+ 1871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:10.035154 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:10.035323 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:10.035397 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:10.035830 (+ 433us) server_negotiation.cc:299] Negotiation successful
0812 02:36:10.035960 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260812 02:36:10.036437 9737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:10.029142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:34685 (local address 127.7.132.130:46509)
0812 02:36:10.030163 (+ 1021us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:10.030180 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:10.030312 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:10.030699 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:10.030703 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:10.030729 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:36:10.031196 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:10.031209 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:10.032293 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:10.032298 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:10.033868 (+ 1570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:10.033884 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:10.035235 (+ 1351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:10.035244 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:10.035485 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:10.035567 (+ 82us) client_negotiation.cc:769] Sending connection context
0812 02:36:10.035743 (+ 176us) client_negotiation.cc:240] Negotiation successful
0812 02:36:10.035894 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":849,"mutex_wait_us":16,"thread_start_us":592,"threads_started":1}
I20260812 02:36:10.036595 9738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:10.029196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.131:41005 (local address 127.7.132.130:42143)
0812 02:36:10.030340 (+ 1144us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:10.030392 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:10.030537 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:10.030911 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:10.030914 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:10.030938 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:36:10.031359 (+ 421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:10.031370 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:10.032436 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:10.032439 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:10.033913 (+ 1474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:10.033926 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:10.035382 (+ 1456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:10.035387 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:10.035582 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:10.035606 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 02:36:10.035775 (+ 169us) client_negotiation.cc:240] Negotiation successful
0812 02:36:10.035905 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":960,"mutex_wait_us":42,"thread_start_us":698,"threads_started":1}
I20260812 02:36:10.037403 9296 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9040dea9bdfb49998886c2d047d46601" candidate_uuid: "be444de7c1c94ee8a92f2a78f03e654f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c7b0dc3dde34ef98a501ea15c294a1b" is_pre_election: true
I20260812 02:36:10.037631 9634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9040dea9bdfb49998886c2d047d46601" candidate_uuid: "be444de7c1c94ee8a92f2a78f03e654f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea68384054b64897b6c13998b543bd1b" is_pre_election: true
I20260812 02:36:10.037945 9296 raft_consensus.cc:2454] T 9040dea9bdfb49998886c2d047d46601 P 3c7b0dc3dde34ef98a501ea15c294a1b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be444de7c1c94ee8a92f2a78f03e654f in term 0.
I20260812 02:36:10.038241 9634 raft_consensus.cc:2454] T 9040dea9bdfb49998886c2d047d46601 P ea68384054b64897b6c13998b543bd1b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be444de7c1c94ee8a92f2a78f03e654f in term 0.
I20260812 02:36:10.038691 9377 leader_election.cc:302] T 9040dea9bdfb49998886c2d047d46601 P be444de7c1c94ee8a92f2a78f03e654f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c7b0dc3dde34ef98a501ea15c294a1b, be444de7c1c94ee8a92f2a78f03e654f; no voters:
I20260812 02:36:10.039057 9733 raft_consensus.cc:2781] T 9040dea9bdfb49998886c2d047d46601 P be444de7c1c94ee8a92f2a78f03e654f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 02:36:10.039235 9733 raft_consensus.cc:482] T 9040dea9bdfb49998886c2d047d46601 P be444de7c1c94ee8a92f2a78f03e654f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 02:36:10.039353 9733 raft_consensus.cc:3037] T 9040dea9bdfb49998886c2d047d46601 P be444de7c1c94ee8a92f2a78f03e654f [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:36:10.041699 9733 raft_consensus.cc:504] T 9040dea9bdfb49998886c2d047d46601 P be444de7c1c94ee8a92f2a78f03e654f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c7b0dc3dde34ef98a501ea15c294a1b" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 34685 } } peers { permanent_uuid: "be444de7c1c94ee8a92f2a78f03e654f" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 42511 } } peers { permanent_uuid: "ea68384054b64897b6c13998b543bd1b" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 41005 } }
I20260812 02:36:10.042438 9733 leader_election.cc:288] T 9040dea9bdfb49998886c2d047d46601 P be444de7c1c94ee8a92f2a78f03e654f [CANDIDATE]: Term 1 election: Requested vote from peers 3c7b0dc3dde34ef98a501ea15c294a1b (127.7.132.129:34685), ea68384054b64897b6c13998b543bd1b (127.7.132.131:41005)
I20260812 02:36:10.042906 9296 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9040dea9bdfb49998886c2d047d46601" candidate_uuid: "be444de7c1c94ee8a92f2a78f03e654f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c7b0dc3dde34ef98a501ea15c294a1b"
I20260812 02:36:10.043016 9634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9040dea9bdfb49998886c2d047d46601" candidate_uuid: "be444de7c1c94ee8a92f2a78f03e654f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea68384054b64897b6c13998b543bd1b"
I20260812 02:36:10.043197 9296 raft_consensus.cc:3037] T 9040dea9bdfb49998886c2d047d46601 P 3c7b0dc3dde34ef98a501ea15c294a1b [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:36:10.043222 9634 raft_consensus.cc:3037] T 9040dea9bdfb49998886c2d047d46601 P ea68384054b64897b6c13998b543bd1b [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:36:10.046612 9634 raft_consensus.cc:2454] T 9040dea9bdfb49998886c2d047d46601 P ea68384054b64897b6c13998b543bd1b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be444de7c1c94ee8a92f2a78f03e654f in term 1.
I20260812 02:36:10.046600 9296 raft_consensus.cc:2454] T 9040dea9bdfb49998886c2d047d46601 P 3c7b0dc3dde34ef98a501ea15c294a1b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be444de7c1c94ee8a92f2a78f03e654f in term 1.
I20260812 02:36:10.047120 9374 leader_election.cc:302] T 9040dea9bdfb49998886c2d047d46601 P be444de7c1c94ee8a92f2a78f03e654f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 3c7b0dc3dde34ef98a501ea15c294a1b, be444de7c1c94ee8a92f2a78f03e654f, ea68384054b64897b6c13998b543bd1b; no voters:
I20260812 02:36:10.047535 9733 raft_consensus.cc:2781] T 9040dea9bdfb49998886c2d047d46601 P be444de7c1c94ee8a92f2a78f03e654f [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:36:10.048059 9733 raft_consensus.cc:686] T 9040dea9bdfb49998886c2d047d46601 P be444de7c1c94ee8a92f2a78f03e654f [term 1 LEADER]: Becoming Leader. State: Replica: be444de7c1c94ee8a92f2a78f03e654f, State: Running, Role: LEADER
I20260812 02:36:10.048641 9733 consensus_queue.cc:237] T 9040dea9bdfb49998886c2d047d46601 P be444de7c1c94ee8a92f2a78f03e654f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c7b0dc3dde34ef98a501ea15c294a1b" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 34685 } } peers { permanent_uuid: "be444de7c1c94ee8a92f2a78f03e654f" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 42511 } } peers { permanent_uuid: "ea68384054b64897b6c13998b543bd1b" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 41005 } }
I20260812 02:36:10.053793 9096 catalog_manager.cc:5495] T 9040dea9bdfb49998886c2d047d46601 P be444de7c1c94ee8a92f2a78f03e654f reported cstate change: term changed from 0 to 1, leader changed from <none> to be444de7c1c94ee8a92f2a78f03e654f (127.7.132.130). New cstate: current_term: 1 leader_uuid: "be444de7c1c94ee8a92f2a78f03e654f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c7b0dc3dde34ef98a501ea15c294a1b" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 34685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be444de7c1c94ee8a92f2a78f03e654f" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 42511 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ea68384054b64897b6c13998b543bd1b" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 41005 } health_report { overall_health: UNKNOWN } } }
I20260812 02:36:10.121563 9723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:10.115695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49638 (local address 127.7.132.130:42511)
0812 02:36:10.115881 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:10.115886 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:10.116106 (+ 220us) server_negotiation.cc:407] Connection header received
0812 02:36:10.116432 (+ 326us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:10.116436 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:10.116517 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:10.116647 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 02:36:10.117155 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:10.117955 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:10.119397 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:10.119810 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:10.119909 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:10.120475 (+ 566us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 02:36:10.121017 (+ 542us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 02:36:10.121129 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:10.121268 (+ 139us) server_negotiation.cc:299] Negotiation successful
0812 02:36:10.121337 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 02:36:10.130908 9634 raft_consensus.cc:1261] T 9040dea9bdfb49998886c2d047d46601 P ea68384054b64897b6c13998b543bd1b [term 1 FOLLOWER]: Refusing update from remote peer be444de7c1c94ee8a92f2a78f03e654f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:36:10.130908 9296 raft_consensus.cc:1261] T 9040dea9bdfb49998886c2d047d46601 P 3c7b0dc3dde34ef98a501ea15c294a1b [term 1 FOLLOWER]: Refusing update from remote peer be444de7c1c94ee8a92f2a78f03e654f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:36:10.132078 9740 consensus_queue.cc:1035] T 9040dea9bdfb49998886c2d047d46601 P be444de7c1c94ee8a92f2a78f03e654f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c7b0dc3dde34ef98a501ea15c294a1b" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 34685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:36:10.132511 9733 consensus_queue.cc:1035] T 9040dea9bdfb49998886c2d047d46601 P be444de7c1c94ee8a92f2a78f03e654f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ea68384054b64897b6c13998b543bd1b" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 41005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:36:10.147260 9743 mvcc.cc:205] Tried to move back new op lower bound from 7317512888844627968 to 7317512888523247616. Current Snapshot: MvccSnapshot[applied={T|T < 7317512888844627968 or (T in {7317512888844627968})}]
I20260812 02:36:10.163933 9095 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50278:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 02:36:10.164358 9095 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50278:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 02:36:10.168850 9095 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 38e2dd3af22545f6b021313a489f7387: Sending DeleteTablet for 3 replicas of tablet 9040dea9bdfb49998886c2d047d46601
I20260812 02:36:10.170267 9429 tablet_service.cc:1514] Processing DeleteTablet for tablet 9040dea9bdfb49998886c2d047d46601 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:36:10 UTC) from {username='kudu', principal='kudu/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:49636
I20260812 02:36:10.170256 9264 tablet_service.cc:1514] Processing DeleteTablet for tablet 9040dea9bdfb49998886c2d047d46601 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:36:10 UTC) from {username='kudu', principal='kudu/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:55008
I20260812 02:36:10.170423 9608 tablet_service.cc:1514] Processing DeleteTablet for tablet 9040dea9bdfb49998886c2d047d46601 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:36:10 UTC) from {username='kudu', principal='kudu/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:54050
I20260812 02:36:10.173877 9095 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50278:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 02:36:10.174620 9095 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 02:36:10.180318 9755 tablet_replica.cc:330] T 9040dea9bdfb49998886c2d047d46601 P ea68384054b64897b6c13998b543bd1b: stopping tablet replica
I20260812 02:36:10.181757 9755 raft_consensus.cc:2229] T 9040dea9bdfb49998886c2d047d46601 P ea68384054b64897b6c13998b543bd1b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 02:36:10.184310 9263 tablet_service.cc:1467] Processing CreateTablet for tablet 5d917e1b434a4b4aa6cfd02f0921ff63 (DEFAULT_TABLE table=test-table [id=8f47d4e40a6b45fbab781ddbb30b0d72]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:36:10.184479 9428 tablet_service.cc:1467] Processing CreateTablet for tablet 5d917e1b434a4b4aa6cfd02f0921ff63 (DEFAULT_TABLE table=test-table [id=8f47d4e40a6b45fbab781ddbb30b0d72]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:36:10.185127 9263 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d917e1b434a4b4aa6cfd02f0921ff63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:36:10.185189 9428 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d917e1b434a4b4aa6cfd02f0921ff63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:36:10.187299 9754 tablet_replica.cc:330] T 9040dea9bdfb49998886c2d047d46601 P be444de7c1c94ee8a92f2a78f03e654f: stopping tablet replica
I20260812 02:36:10.183308 9755 raft_consensus.cc:2258] T 9040dea9bdfb49998886c2d047d46601 P ea68384054b64897b6c13998b543bd1b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:36:10.189004 9754 raft_consensus.cc:2229] T 9040dea9bdfb49998886c2d047d46601 P be444de7c1c94ee8a92f2a78f03e654f [term 1 LEADER]: Raft consensus shutting down.
I20260812 02:36:10.189812 9754 raft_consensus.cc:2258] T 9040dea9bdfb49998886c2d047d46601 P be444de7c1c94ee8a92f2a78f03e654f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:36:10.186619 9607 tablet_service.cc:1467] Processing CreateTablet for tablet 5d917e1b434a4b4aa6cfd02f0921ff63 (DEFAULT_TABLE table=test-table [id=8f47d4e40a6b45fbab781ddbb30b0d72]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:36:10.190805 9607 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d917e1b434a4b4aa6cfd02f0921ff63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:36:10.191901 9752 tablet_replica.cc:330] T 9040dea9bdfb49998886c2d047d46601 P 3c7b0dc3dde34ef98a501ea15c294a1b: stopping tablet replica
I20260812 02:36:10.192371 9752 raft_consensus.cc:2229] T 9040dea9bdfb49998886c2d047d46601 P 3c7b0dc3dde34ef98a501ea15c294a1b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 02:36:10.192579 9726 tablet_bootstrap.cc:492] T 5d917e1b434a4b4aa6cfd02f0921ff63 P be444de7c1c94ee8a92f2a78f03e654f: Bootstrap starting.
I20260812 02:36:10.192936 9752 raft_consensus.cc:2258] T 9040dea9bdfb49998886c2d047d46601 P 3c7b0dc3dde34ef98a501ea15c294a1b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:36:10.194309 9726 tablet_bootstrap.cc:654] T 5d917e1b434a4b4aa6cfd02f0921ff63 P be444de7c1c94ee8a92f2a78f03e654f: Neither blocks nor log segments found. Creating new log.
I20260812 02:36:10.196237 9728 tablet_bootstrap.cc:492] T 5d917e1b434a4b4aa6cfd02f0921ff63 P 3c7b0dc3dde34ef98a501ea15c294a1b: Bootstrap starting.
I20260812 02:36:10.196645 9754 ts_tablet_manager.cc:1905] T 9040dea9bdfb49998886c2d047d46601 P be444de7c1c94ee8a92f2a78f03e654f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:36:10.197021 9755 ts_tablet_manager.cc:1905] T 9040dea9bdfb49998886c2d047d46601 P ea68384054b64897b6c13998b543bd1b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:36:10.197891 9728 tablet_bootstrap.cc:654] T 5d917e1b434a4b4aa6cfd02f0921ff63 P 3c7b0dc3dde34ef98a501ea15c294a1b: Neither blocks nor log segments found. Creating new log.
I20260812 02:36:10.202432 9727 tablet_bootstrap.cc:492] T 5d917e1b434a4b4aa6cfd02f0921ff63 P ea68384054b64897b6c13998b543bd1b: Bootstrap starting.
I20260812 02:36:10.204162 9727 tablet_bootstrap.cc:654] T 5d917e1b434a4b4aa6cfd02f0921ff63 P ea68384054b64897b6c13998b543bd1b: Neither blocks nor log segments found. Creating new log.
I20260812 02:36:10.205071 9752 ts_tablet_manager.cc:1905] T 9040dea9bdfb49998886c2d047d46601 P 3c7b0dc3dde34ef98a501ea15c294a1b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:36:10.211776 9755 ts_tablet_manager.cc:1918] T 9040dea9bdfb49998886c2d047d46601 P ea68384054b64897b6c13998b543bd1b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 02:36:10.212020 9755 log.cc:1196] T 9040dea9bdfb49998886c2d047d46601 P ea68384054b64897b6c13998b543bd1b: Deleting WAL directory at /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502156971023-7698-0/minicluster-data/ts-2/wal/wals/9040dea9bdfb49998886c2d047d46601
I20260812 02:36:10.212656 9755 ts_tablet_manager.cc:1939] T 9040dea9bdfb49998886c2d047d46601 P ea68384054b64897b6c13998b543bd1b: Deleting consensus metadata
I20260812 02:36:10.214905 9077 catalog_manager.cc:4841] TS ea68384054b64897b6c13998b543bd1b (127.7.132.131:41005): tablet 9040dea9bdfb49998886c2d047d46601 (table test-table [id=a79546a4eed84eb89aa88c0b6f0de479]) successfully deleted
I20260812 02:36:10.217730 9752 ts_tablet_manager.cc:1918] T 9040dea9bdfb49998886c2d047d46601 P 3c7b0dc3dde34ef98a501ea15c294a1b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 02:36:10.217897 9752 log.cc:1196] T 9040dea9bdfb49998886c2d047d46601 P 3c7b0dc3dde34ef98a501ea15c294a1b: Deleting WAL directory at /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502156971023-7698-0/minicluster-data/ts-0/wal/wals/9040dea9bdfb49998886c2d047d46601
I20260812 02:36:10.218427 9752 ts_tablet_manager.cc:1939] T 9040dea9bdfb49998886c2d047d46601 P 3c7b0dc3dde34ef98a501ea15c294a1b: Deleting consensus metadata
I20260812 02:36:10.220038 9079 catalog_manager.cc:4841] TS 3c7b0dc3dde34ef98a501ea15c294a1b (127.7.132.129:34685): tablet 9040dea9bdfb49998886c2d047d46601 (table test-table [id=a79546a4eed84eb89aa88c0b6f0de479]) successfully deleted
I20260812 02:36:10.224802 9754 ts_tablet_manager.cc:1918] T 9040dea9bdfb49998886c2d047d46601 P be444de7c1c94ee8a92f2a78f03e654f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 02:36:10.225050 9754 log.cc:1196] T 9040dea9bdfb49998886c2d047d46601 P be444de7c1c94ee8a92f2a78f03e654f: Deleting WAL directory at /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502156971023-7698-0/minicluster-data/ts-1/wal/wals/9040dea9bdfb49998886c2d047d46601
I20260812 02:36:10.226262 9727 tablet_bootstrap.cc:492] T 5d917e1b434a4b4aa6cfd02f0921ff63 P ea68384054b64897b6c13998b543bd1b: No bootstrap required, opened a new log
I20260812 02:36:10.226455 9727 ts_tablet_manager.cc:1397] T 5d917e1b434a4b4aa6cfd02f0921ff63 P ea68384054b64897b6c13998b543bd1b: Time spent bootstrapping tablet: real 0.024s user 0.001s sys 0.009s
I20260812 02:36:10.227259 9727 raft_consensus.cc:348] T 5d917e1b434a4b4aa6cfd02f0921ff63 P ea68384054b64897b6c13998b543bd1b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c7b0dc3dde34ef98a501ea15c294a1b" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 34685 } } peers { permanent_uuid: "be444de7c1c94ee8a92f2a78f03e654f" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 42511 } } peers { permanent_uuid: "ea68384054b64897b6c13998b543bd1b" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 41005 } }
I20260812 02:36:10.227582 9727 raft_consensus.cc:374] T 5d917e1b434a4b4aa6cfd02f0921ff63 P ea68384054b64897b6c13998b543bd1b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:36:10.227726 9727 raft_consensus.cc:729] T 5d917e1b434a4b4aa6cfd02f0921ff63 P ea68384054b64897b6c13998b543bd1b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea68384054b64897b6c13998b543bd1b, State: Initialized, Role: FOLLOWER
I20260812 02:36:10.228046 9727 consensus_queue.cc:260] T 5d917e1b434a4b4aa6cfd02f0921ff63 P ea68384054b64897b6c13998b543bd1b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c7b0dc3dde34ef98a501ea15c294a1b" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 34685 } } peers { permanent_uuid: "be444de7c1c94ee8a92f2a78f03e654f" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 42511 } } peers { permanent_uuid: "ea68384054b64897b6c13998b543bd1b" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 41005 } }
I20260812 02:36:10.228742 9727 ts_tablet_manager.cc:1428] T 5d917e1b434a4b4aa6cfd02f0921ff63 P ea68384054b64897b6c13998b543bd1b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 02:36:10.230402 9728 tablet_bootstrap.cc:492] T 5d917e1b434a4b4aa6cfd02f0921ff63 P 3c7b0dc3dde34ef98a501ea15c294a1b: No bootstrap required, opened a new log
I20260812 02:36:10.230598 9728 ts_tablet_manager.cc:1397] T 5d917e1b434a4b4aa6cfd02f0921ff63 P 3c7b0dc3dde34ef98a501ea15c294a1b: Time spent bootstrapping tablet: real 0.034s user 0.003s sys 0.009s
I20260812 02:36:10.230808 9754 ts_tablet_manager.cc:1939] T 9040dea9bdfb49998886c2d047d46601 P be444de7c1c94ee8a92f2a78f03e654f: Deleting consensus metadata
I20260812 02:36:10.231513 9728 raft_consensus.cc:348] T 5d917e1b434a4b4aa6cfd02f0921ff63 P 3c7b0dc3dde34ef98a501ea15c294a1b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c7b0dc3dde34ef98a501ea15c294a1b" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 34685 } } peers { permanent_uuid: "be444de7c1c94ee8a92f2a78f03e654f" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 42511 } } peers { permanent_uuid: "ea68384054b64897b6c13998b543bd1b" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 41005 } }
I20260812 02:36:10.231808 9728 raft_consensus.cc:374] T 5d917e1b434a4b4aa6cfd02f0921ff63 P 3c7b0dc3dde34ef98a501ea15c294a1b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:36:10.231948 9728 raft_consensus.cc:729] T 5d917e1b434a4b4aa6cfd02f0921ff63 P 3c7b0dc3dde34ef98a501ea15c294a1b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c7b0dc3dde34ef98a501ea15c294a1b, State: Initialized, Role: FOLLOWER
I20260812 02:36:10.232282 9728 consensus_queue.cc:260] T 5d917e1b434a4b4aa6cfd02f0921ff63 P 3c7b0dc3dde34ef98a501ea15c294a1b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c7b0dc3dde34ef98a501ea15c294a1b" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 34685 } } peers { permanent_uuid: "be444de7c1c94ee8a92f2a78f03e654f" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 42511 } } peers { permanent_uuid: "ea68384054b64897b6c13998b543bd1b" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 41005 } }
I20260812 02:36:10.232731 9075 catalog_manager.cc:4841] TS be444de7c1c94ee8a92f2a78f03e654f (127.7.132.130:42511): tablet 9040dea9bdfb49998886c2d047d46601 (table test-table [id=a79546a4eed84eb89aa88c0b6f0de479]) successfully deleted
I20260812 02:36:10.233100 9728 ts_tablet_manager.cc:1428] T 5d917e1b434a4b4aa6cfd02f0921ff63 P 3c7b0dc3dde34ef98a501ea15c294a1b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 02:36:10.235932 9726 tablet_bootstrap.cc:492] T 5d917e1b434a4b4aa6cfd02f0921ff63 P be444de7c1c94ee8a92f2a78f03e654f: No bootstrap required, opened a new log
I20260812 02:36:10.236104 9726 ts_tablet_manager.cc:1397] T 5d917e1b434a4b4aa6cfd02f0921ff63 P be444de7c1c94ee8a92f2a78f03e654f: Time spent bootstrapping tablet: real 0.044s user 0.018s sys 0.019s
I20260812 02:36:10.236969 9726 raft_consensus.cc:348] T 5d917e1b434a4b4aa6cfd02f0921ff63 P be444de7c1c94ee8a92f2a78f03e654f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c7b0dc3dde34ef98a501ea15c294a1b" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 34685 } } peers { permanent_uuid: "be444de7c1c94ee8a92f2a78f03e654f" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 42511 } } peers { permanent_uuid: "ea68384054b64897b6c13998b543bd1b" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 41005 } }
I20260812 02:36:10.237214 9726 raft_consensus.cc:374] T 5d917e1b434a4b4aa6cfd02f0921ff63 P be444de7c1c94ee8a92f2a78f03e654f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:36:10.237332 9726 raft_consensus.cc:729] T 5d917e1b434a4b4aa6cfd02f0921ff63 P be444de7c1c94ee8a92f2a78f03e654f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be444de7c1c94ee8a92f2a78f03e654f, State: Initialized, Role: FOLLOWER
I20260812 02:36:10.237630 9726 consensus_queue.cc:260] T 5d917e1b434a4b4aa6cfd02f0921ff63 P be444de7c1c94ee8a92f2a78f03e654f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c7b0dc3dde34ef98a501ea15c294a1b" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 34685 } } peers { permanent_uuid: "be444de7c1c94ee8a92f2a78f03e654f" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 42511 } } peers { permanent_uuid: "ea68384054b64897b6c13998b543bd1b" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 41005 } }
I20260812 02:36:10.238353 9726 ts_tablet_manager.cc:1428] T 5d917e1b434a4b4aa6cfd02f0921ff63 P be444de7c1c94ee8a92f2a78f03e654f: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260812 02:36:10.290252 9734 raft_consensus.cc:482] T 5d917e1b434a4b4aa6cfd02f0921ff63 P 3c7b0dc3dde34ef98a501ea15c294a1b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 02:36:10.290549 9734 raft_consensus.cc:504] T 5d917e1b434a4b4aa6cfd02f0921ff63 P 3c7b0dc3dde34ef98a501ea15c294a1b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c7b0dc3dde34ef98a501ea15c294a1b" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 34685 } } peers { permanent_uuid: "be444de7c1c94ee8a92f2a78f03e654f" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 42511 } } peers { permanent_uuid: "ea68384054b64897b6c13998b543bd1b" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 41005 } }
I20260812 02:36:10.292168 9734 leader_election.cc:288] T 5d917e1b434a4b4aa6cfd02f0921ff63 P 3c7b0dc3dde34ef98a501ea15c294a1b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be444de7c1c94ee8a92f2a78f03e654f (127.7.132.130:42511), ea68384054b64897b6c13998b543bd1b (127.7.132.131:41005)
I20260812 02:36:10.300688 9723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:10.292758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:59991 (local address 127.7.132.130:42511)
0812 02:36:10.293003 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:10.293012 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:10.293674 (+ 662us) server_negotiation.cc:407] Connection header received
0812 02:36:10.294047 (+ 373us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:10.294054 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:10.294181 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:10.294294 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:36:10.295256 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:10.296372 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:10.298305 (+ 1933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:10.299590 (+ 1285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:10.299759 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:10.299846 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:10.300213 (+ 367us) server_negotiation.cc:299] Negotiation successful
0812 02:36:10.300346 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260812 02:36:10.301008 9759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:10.292514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.130:42511 (local address 127.7.132.129:59991)
0812 02:36:10.293542 (+ 1028us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:10.293565 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:10.293820 (+ 255us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:10.294484 (+ 664us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:10.294489 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:10.294550 (+ 61us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:36:10.295065 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:10.295078 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:10.296505 (+ 1427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:10.296510 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:10.298147 (+ 1637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:10.298169 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:10.299826 (+ 1657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:10.299832 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:10.300019 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:10.300056 (+ 37us) client_negotiation.cc:769] Sending connection context
0812 02:36:10.300163 (+ 107us) client_negotiation.cc:240] Negotiation successful
0812 02:36:10.300304 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":862,"mutex_wait_us":40,"spinlock_wait_cycles":7040,"thread_start_us":658,"threads_started":1}
I20260812 02:36:10.301358 9725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:10.293168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:58489 (local address 127.7.132.131:41005)
0812 02:36:10.293339 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:10.293345 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:10.293965 (+ 620us) server_negotiation.cc:407] Connection header received
0812 02:36:10.294282 (+ 317us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:10.294287 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:10.294351 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:10.294451 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:36:10.295055 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:10.296187 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:10.298959 (+ 2772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:10.300109 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:10.300280 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:10.300348 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:10.300950 (+ 602us) server_negotiation.cc:299] Negotiation successful
0812 02:36:10.301096 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 02:36:10.302255 9760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:10.292704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.131:41005 (local address 127.7.132.129:58489)
0812 02:36:10.293873 (+ 1169us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:10.293891 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:10.294072 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:10.294478 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:10.294483 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:10.294509 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:36:10.294880 (+ 371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:10.294890 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:10.296346 (+ 1456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:10.296352 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:10.298741 (+ 2389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:10.298757 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:10.300533 (+ 1776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:10.300539 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:10.300752 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:10.300784 (+ 32us) client_negotiation.cc:769] Sending connection context
0812 02:36:10.301621 (+ 837us) client_negotiation.cc:240] Negotiation successful
0812 02:36:10.301792 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1051,"mutex_wait_us":57,"thread_start_us":843,"threads_started":1}
I20260812 02:36:10.302167 9461 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d917e1b434a4b4aa6cfd02f0921ff63" candidate_uuid: "3c7b0dc3dde34ef98a501ea15c294a1b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be444de7c1c94ee8a92f2a78f03e654f" is_pre_election: true
I20260812 02:36:10.302726 9461 raft_consensus.cc:2454] T 5d917e1b434a4b4aa6cfd02f0921ff63 P be444de7c1c94ee8a92f2a78f03e654f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c7b0dc3dde34ef98a501ea15c294a1b in term 0.
I20260812 02:36:10.303025 9634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d917e1b434a4b4aa6cfd02f0921ff63" candidate_uuid: "3c7b0dc3dde34ef98a501ea15c294a1b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea68384054b64897b6c13998b543bd1b" is_pre_election: true
I20260812 02:36:10.303313 9634 raft_consensus.cc:2454] T 5d917e1b434a4b4aa6cfd02f0921ff63 P ea68384054b64897b6c13998b543bd1b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c7b0dc3dde34ef98a501ea15c294a1b in term 0.
I20260812 02:36:10.303532 9206 leader_election.cc:302] T 5d917e1b434a4b4aa6cfd02f0921ff63 P 3c7b0dc3dde34ef98a501ea15c294a1b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c7b0dc3dde34ef98a501ea15c294a1b, be444de7c1c94ee8a92f2a78f03e654f; no voters:
I20260812 02:36:10.304018 9734 raft_consensus.cc:2781] T 5d917e1b434a4b4aa6cfd02f0921ff63 P 3c7b0dc3dde34ef98a501ea15c294a1b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 02:36:10.304155 9734 raft_consensus.cc:482] T 5d917e1b434a4b4aa6cfd02f0921ff63 P 3c7b0dc3dde34ef98a501ea15c294a1b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 02:36:10.304255 9734 raft_consensus.cc:3037] T 5d917e1b434a4b4aa6cfd02f0921ff63 P 3c7b0dc3dde34ef98a501ea15c294a1b [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:36:10.306540 9734 raft_consensus.cc:504] T 5d917e1b434a4b4aa6cfd02f0921ff63 P 3c7b0dc3dde34ef98a501ea15c294a1b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c7b0dc3dde34ef98a501ea15c294a1b" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 34685 } } peers { permanent_uuid: "be444de7c1c94ee8a92f2a78f03e654f" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 42511 } } peers { permanent_uuid: "ea68384054b64897b6c13998b543bd1b" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 41005 } }
I20260812 02:36:10.307184 9734 leader_election.cc:288] T 5d917e1b434a4b4aa6cfd02f0921ff63 P 3c7b0dc3dde34ef98a501ea15c294a1b [CANDIDATE]: Term 1 election: Requested vote from peers be444de7c1c94ee8a92f2a78f03e654f (127.7.132.130:42511), ea68384054b64897b6c13998b543bd1b (127.7.132.131:41005)
I20260812 02:36:10.307684 9461 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d917e1b434a4b4aa6cfd02f0921ff63" candidate_uuid: "3c7b0dc3dde34ef98a501ea15c294a1b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be444de7c1c94ee8a92f2a78f03e654f"
I20260812 02:36:10.307735 9634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d917e1b434a4b4aa6cfd02f0921ff63" candidate_uuid: "3c7b0dc3dde34ef98a501ea15c294a1b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea68384054b64897b6c13998b543bd1b"
I20260812 02:36:10.307898 9461 raft_consensus.cc:3037] T 5d917e1b434a4b4aa6cfd02f0921ff63 P be444de7c1c94ee8a92f2a78f03e654f [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:36:10.307969 9634 raft_consensus.cc:3037] T 5d917e1b434a4b4aa6cfd02f0921ff63 P ea68384054b64897b6c13998b543bd1b [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:36:10.310276 9461 raft_consensus.cc:2454] T 5d917e1b434a4b4aa6cfd02f0921ff63 P be444de7c1c94ee8a92f2a78f03e654f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c7b0dc3dde34ef98a501ea15c294a1b in term 1.
I20260812 02:36:10.310686 9206 leader_election.cc:302] T 5d917e1b434a4b4aa6cfd02f0921ff63 P 3c7b0dc3dde34ef98a501ea15c294a1b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c7b0dc3dde34ef98a501ea15c294a1b, be444de7c1c94ee8a92f2a78f03e654f; no voters:
I20260812 02:36:10.310977 9734 raft_consensus.cc:2781] T 5d917e1b434a4b4aa6cfd02f0921ff63 P 3c7b0dc3dde34ef98a501ea15c294a1b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:36:10.311023 9634 raft_consensus.cc:2454] T 5d917e1b434a4b4aa6cfd02f0921ff63 P ea68384054b64897b6c13998b543bd1b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c7b0dc3dde34ef98a501ea15c294a1b in term 1.
I20260812 02:36:10.311520 9734 raft_consensus.cc:686] T 5d917e1b434a4b4aa6cfd02f0921ff63 P 3c7b0dc3dde34ef98a501ea15c294a1b [term 1 LEADER]: Becoming Leader. State: Replica: 3c7b0dc3dde34ef98a501ea15c294a1b, State: Running, Role: LEADER
I20260812 02:36:10.312129 9734 consensus_queue.cc:237] T 5d917e1b434a4b4aa6cfd02f0921ff63 P 3c7b0dc3dde34ef98a501ea15c294a1b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c7b0dc3dde34ef98a501ea15c294a1b" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 34685 } } peers { permanent_uuid: "be444de7c1c94ee8a92f2a78f03e654f" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 42511 } } peers { permanent_uuid: "ea68384054b64897b6c13998b543bd1b" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 41005 } }
I20260812 02:36:10.315610 9095 catalog_manager.cc:5495] T 5d917e1b434a4b4aa6cfd02f0921ff63 P 3c7b0dc3dde34ef98a501ea15c294a1b reported cstate change: term changed from 0 to 1, leader changed from <none> to 3c7b0dc3dde34ef98a501ea15c294a1b (127.7.132.129). New cstate: current_term: 1 leader_uuid: "3c7b0dc3dde34ef98a501ea15c294a1b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c7b0dc3dde34ef98a501ea15c294a1b" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 34685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "be444de7c1c94ee8a92f2a78f03e654f" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 42511 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ea68384054b64897b6c13998b543bd1b" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 41005 } health_report { overall_health: UNKNOWN } } }
Aug 12 02:36:10 dist-test-slave-mn4r krb5kdc[9040](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502167, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.190@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 02:36:10.386451 9773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:10.368896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:43261 (local address 127.0.0.1:50286)
0812 02:36:10.369868 (+ 972us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:10.369889 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:10.370052 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:10.370441 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:10.370448 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:10.371456 (+ 1008us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:36:10.371740 (+ 284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:10.371753 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:10.372861 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:10.372867 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:10.374362 (+ 1495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:10.374378 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:10.375060 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:10.375066 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:10.375263 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:10.376024 (+ 761us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:36:10.376049 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:36:10.378895 (+ 2846us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:36:10.382719 (+ 3824us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:10.382732 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:10.382741 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:10.383393 (+ 652us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:10.384011 (+ 618us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:10.384020 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:10.384024 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:10.384259 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:10.385051 (+ 792us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:36:10.385060 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:36:10.385362 (+ 302us) client_negotiation.cc:769] Sending connection context
0812 02:36:10.385585 (+ 223us) client_negotiation.cc:240] Negotiation successful
0812 02:36:10.385897 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":796,"thread_start_us":632,"threads_started":1}
I20260812 02:36:10.386471 9770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:10.368866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50286 (local address 127.7.132.190:43261)
0812 02:36:10.369679 (+ 813us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:10.369689 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:10.369990 (+ 301us) server_negotiation.cc:407] Connection header received
0812 02:36:10.370204 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:10.370208 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:10.370279 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:10.370372 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:10.371918 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:10.372726 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:10.374502 (+ 1776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:10.374905 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:10.375025 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:10.379100 (+ 4075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:10.379105 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:10.379108 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:10.379139 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:10.382567 (+ 3428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:10.383585 (+ 1018us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:10.383591 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:10.383595 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:10.383751 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:10.384426 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:10.384433 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:10.384436 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:10.384902 (+ 466us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:10.385067 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:10.385707 (+ 640us) server_negotiation.cc:299] Negotiation successful
0812 02:36:10.386048 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":155,"server-negotiator.queue_time_us":627,"thread_start_us":384,"threads_started":1}
I20260812 02:36:10.392468 9095 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:50286:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260812 02:36:10.393795 9095 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 02:36:10.400766 9263 tablet_service.cc:1467] Processing CreateTablet for tablet 9980a009cc164788ab918d402249ab6d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a7aec3b8cf9745d4b13d3f1401c58803]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 02:36:10.401273 9263 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9980a009cc164788ab918d402249ab6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:36:10.401120 9428 tablet_service.cc:1467] Processing CreateTablet for tablet 9980a009cc164788ab918d402249ab6d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a7aec3b8cf9745d4b13d3f1401c58803]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 02:36:10.401652 9428 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9980a009cc164788ab918d402249ab6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:36:10.406865 9728 tablet_bootstrap.cc:492] T 9980a009cc164788ab918d402249ab6d P 3c7b0dc3dde34ef98a501ea15c294a1b: Bootstrap starting.
I20260812 02:36:10.406667 9607 tablet_service.cc:1467] Processing CreateTablet for tablet 9980a009cc164788ab918d402249ab6d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a7aec3b8cf9745d4b13d3f1401c58803]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 02:36:10.407186 9607 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9980a009cc164788ab918d402249ab6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:36:10.408558 9728 tablet_bootstrap.cc:654] T 9980a009cc164788ab918d402249ab6d P 3c7b0dc3dde34ef98a501ea15c294a1b: Neither blocks nor log segments found. Creating new log.
I20260812 02:36:10.408910 9726 tablet_bootstrap.cc:492] T 9980a009cc164788ab918d402249ab6d P be444de7c1c94ee8a92f2a78f03e654f: Bootstrap starting.
I20260812 02:36:10.410595 9726 tablet_bootstrap.cc:654] T 9980a009cc164788ab918d402249ab6d P be444de7c1c94ee8a92f2a78f03e654f: Neither blocks nor log segments found. Creating new log.
W20260812 02:36:10.416287 9350 tablet.cc:2369] T 5d917e1b434a4b4aa6cfd02f0921ff63 P 3c7b0dc3dde34ef98a501ea15c294a1b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 02:36:10.420922 9727 tablet_bootstrap.cc:492] T 9980a009cc164788ab918d402249ab6d P ea68384054b64897b6c13998b543bd1b: Bootstrap starting.
I20260812 02:36:10.421401 9728 tablet_bootstrap.cc:492] T 9980a009cc164788ab918d402249ab6d P 3c7b0dc3dde34ef98a501ea15c294a1b: No bootstrap required, opened a new log
I20260812 02:36:10.421566 9728 ts_tablet_manager.cc:1397] T 9980a009cc164788ab918d402249ab6d P 3c7b0dc3dde34ef98a501ea15c294a1b: Time spent bootstrapping tablet: real 0.015s user 0.001s sys 0.011s
I20260812 02:36:10.422416 9728 raft_consensus.cc:348] T 9980a009cc164788ab918d402249ab6d P 3c7b0dc3dde34ef98a501ea15c294a1b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c7b0dc3dde34ef98a501ea15c294a1b" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 34685 } } peers { permanent_uuid: "be444de7c1c94ee8a92f2a78f03e654f" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 42511 } } peers { permanent_uuid: "ea68384054b64897b6c13998b543bd1b" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 41005 } }
I20260812 02:36:10.422700 9728 raft_consensus.cc:374] T 9980a009cc164788ab918d402249ab6d P 3c7b0dc3dde34ef98a501ea15c294a1b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:36:10.422785 9728 raft_consensus.cc:729] T 9980a009cc164788ab918d402249ab6d P 3c7b0dc3dde34ef98a501ea15c294a1b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c7b0dc3dde34ef98a501ea15c294a1b, State: Initialized, Role: FOLLOWER
I20260812 02:36:10.422806 9727 tablet_bootstrap.cc:654] T 9980a009cc164788ab918d402249ab6d P ea68384054b64897b6c13998b543bd1b: Neither blocks nor log segments found. Creating new log.
I20260812 02:36:10.423108 9728 consensus_queue.cc:260] T 9980a009cc164788ab918d402249ab6d P 3c7b0dc3dde34ef98a501ea15c294a1b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c7b0dc3dde34ef98a501ea15c294a1b" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 34685 } } peers { permanent_uuid: "be444de7c1c94ee8a92f2a78f03e654f" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 42511 } } peers { permanent_uuid: "ea68384054b64897b6c13998b543bd1b" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 41005 } }
I20260812 02:36:10.423267 9726 tablet_bootstrap.cc:492] T 9980a009cc164788ab918d402249ab6d P be444de7c1c94ee8a92f2a78f03e654f: No bootstrap required, opened a new log
I20260812 02:36:10.423466 9726 ts_tablet_manager.cc:1397] T 9980a009cc164788ab918d402249ab6d P be444de7c1c94ee8a92f2a78f03e654f: Time spent bootstrapping tablet: real 0.015s user 0.012s sys 0.000s
I20260812 02:36:10.423641 9734 tablet_replica.cc:439] T 9980a009cc164788ab918d402249ab6d P 3c7b0dc3dde34ef98a501ea15c294a1b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c7b0dc3dde34ef98a501ea15c294a1b" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 34685 } } peers { permanent_uuid: "be444de7c1c94ee8a92f2a78f03e654f" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 42511 } } peers { permanent_uuid: "ea68384054b64897b6c13998b543bd1b" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 41005 } } }
I20260812 02:36:10.423939 9734 tablet_replica.cc:442] T 9980a009cc164788ab918d402249ab6d P 3c7b0dc3dde34ef98a501ea15c294a1b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 02:36:10.424242 9726 raft_consensus.cc:348] T 9980a009cc164788ab918d402249ab6d P be444de7c1c94ee8a92f2a78f03e654f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c7b0dc3dde34ef98a501ea15c294a1b" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 34685 } } peers { permanent_uuid: "be444de7c1c94ee8a92f2a78f03e654f" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 42511 } } peers { permanent_uuid: "ea68384054b64897b6c13998b543bd1b" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 41005 } }
I20260812 02:36:10.424582 9726 raft_consensus.cc:374] T 9980a009cc164788ab918d402249ab6d P be444de7c1c94ee8a92f2a78f03e654f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:36:10.424671 9726 raft_consensus.cc:729] T 9980a009cc164788ab918d402249ab6d P be444de7c1c94ee8a92f2a78f03e654f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be444de7c1c94ee8a92f2a78f03e654f, State: Initialized, Role: FOLLOWER
I20260812 02:36:10.425197 9728 ts_tablet_manager.cc:1428] T 9980a009cc164788ab918d402249ab6d P 3c7b0dc3dde34ef98a501ea15c294a1b: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260812 02:36:10.426339 9726 consensus_queue.cc:260] T 9980a009cc164788ab918d402249ab6d P be444de7c1c94ee8a92f2a78f03e654f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c7b0dc3dde34ef98a501ea15c294a1b" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 34685 } } peers { permanent_uuid: "be444de7c1c94ee8a92f2a78f03e654f" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 42511 } } peers { permanent_uuid: "ea68384054b64897b6c13998b543bd1b" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 41005 } }
I20260812 02:36:10.426816 9733 tablet_replica.cc:439] T 9980a009cc164788ab918d402249ab6d P be444de7c1c94ee8a92f2a78f03e654f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c7b0dc3dde34ef98a501ea15c294a1b" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 34685 } } peers { permanent_uuid: "be444de7c1c94ee8a92f2a78f03e654f" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 42511 } } peers { permanent_uuid: "ea68384054b64897b6c13998b543bd1b" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 41005 } } }
I20260812 02:36:10.427088 9733 tablet_replica.cc:442] T 9980a009cc164788ab918d402249ab6d P be444de7c1c94ee8a92f2a78f03e654f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 02:36:10.430117 9726 ts_tablet_manager.cc:1428] T 9980a009cc164788ab918d402249ab6d P be444de7c1c94ee8a92f2a78f03e654f: Time spent starting tablet: real 0.007s user 0.002s sys 0.001s
I20260812 02:36:10.434896 9727 tablet_bootstrap.cc:492] T 9980a009cc164788ab918d402249ab6d P ea68384054b64897b6c13998b543bd1b: No bootstrap required, opened a new log
I20260812 02:36:10.435111 9727 ts_tablet_manager.cc:1397] T 9980a009cc164788ab918d402249ab6d P ea68384054b64897b6c13998b543bd1b: Time spent bootstrapping tablet: real 0.014s user 0.007s sys 0.003s
I20260812 02:36:10.435936 9727 raft_consensus.cc:348] T 9980a009cc164788ab918d402249ab6d P ea68384054b64897b6c13998b543bd1b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c7b0dc3dde34ef98a501ea15c294a1b" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 34685 } } peers { permanent_uuid: "be444de7c1c94ee8a92f2a78f03e654f" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 42511 } } peers { permanent_uuid: "ea68384054b64897b6c13998b543bd1b" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 41005 } }
I20260812 02:36:10.436167 9727 raft_consensus.cc:374] T 9980a009cc164788ab918d402249ab6d P ea68384054b64897b6c13998b543bd1b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:36:10.436265 9727 raft_consensus.cc:729] T 9980a009cc164788ab918d402249ab6d P ea68384054b64897b6c13998b543bd1b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea68384054b64897b6c13998b543bd1b, State: Initialized, Role: FOLLOWER
I20260812 02:36:10.436553 9727 consensus_queue.cc:260] T 9980a009cc164788ab918d402249ab6d P ea68384054b64897b6c13998b543bd1b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c7b0dc3dde34ef98a501ea15c294a1b" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 34685 } } peers { permanent_uuid: "be444de7c1c94ee8a92f2a78f03e654f" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 42511 } } peers { permanent_uuid: "ea68384054b64897b6c13998b543bd1b" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 41005 } }
I20260812 02:36:10.437119 9727 ts_tablet_manager.cc:1428] T 9980a009cc164788ab918d402249ab6d P ea68384054b64897b6c13998b543bd1b: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260812 02:36:10.437026 9735 tablet_replica.cc:439] T 9980a009cc164788ab918d402249ab6d P ea68384054b64897b6c13998b543bd1b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c7b0dc3dde34ef98a501ea15c294a1b" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 34685 } } peers { permanent_uuid: "be444de7c1c94ee8a92f2a78f03e654f" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 42511 } } peers { permanent_uuid: "ea68384054b64897b6c13998b543bd1b" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 41005 } } }
I20260812 02:36:10.437393 9735 tablet_replica.cc:442] T 9980a009cc164788ab918d402249ab6d P ea68384054b64897b6c13998b543bd1b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 02:36:10.445561 9733 raft_consensus.cc:482] T 9980a009cc164788ab918d402249ab6d P be444de7c1c94ee8a92f2a78f03e654f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 02:36:10.445752 9733 raft_consensus.cc:504] T 9980a009cc164788ab918d402249ab6d P be444de7c1c94ee8a92f2a78f03e654f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c7b0dc3dde34ef98a501ea15c294a1b" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 34685 } } peers { permanent_uuid: "be444de7c1c94ee8a92f2a78f03e654f" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 42511 } } peers { permanent_uuid: "ea68384054b64897b6c13998b543bd1b" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 41005 } }
I20260812 02:36:10.446601 9733 leader_election.cc:288] T 9980a009cc164788ab918d402249ab6d P be444de7c1c94ee8a92f2a78f03e654f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c7b0dc3dde34ef98a501ea15c294a1b (127.7.132.129:34685), ea68384054b64897b6c13998b543bd1b (127.7.132.131:41005)
I20260812 02:36:10.447104 9296 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9980a009cc164788ab918d402249ab6d" candidate_uuid: "be444de7c1c94ee8a92f2a78f03e654f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c7b0dc3dde34ef98a501ea15c294a1b" is_pre_election: true
I20260812 02:36:10.447103 9634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9980a009cc164788ab918d402249ab6d" candidate_uuid: "be444de7c1c94ee8a92f2a78f03e654f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea68384054b64897b6c13998b543bd1b" is_pre_election: true
I20260812 02:36:10.447391 9296 raft_consensus.cc:2454] T 9980a009cc164788ab918d402249ab6d P 3c7b0dc3dde34ef98a501ea15c294a1b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be444de7c1c94ee8a92f2a78f03e654f in term 0.
I20260812 02:36:10.447412 9634 raft_consensus.cc:2454] T 9980a009cc164788ab918d402249ab6d P ea68384054b64897b6c13998b543bd1b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be444de7c1c94ee8a92f2a78f03e654f in term 0.
I20260812 02:36:10.447839 9374 leader_election.cc:302] T 9980a009cc164788ab918d402249ab6d P be444de7c1c94ee8a92f2a78f03e654f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be444de7c1c94ee8a92f2a78f03e654f, ea68384054b64897b6c13998b543bd1b; no voters:
I20260812 02:36:10.448135 9733 raft_consensus.cc:2781] T 9980a009cc164788ab918d402249ab6d P be444de7c1c94ee8a92f2a78f03e654f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 02:36:10.448355 9733 raft_consensus.cc:482] T 9980a009cc164788ab918d402249ab6d P be444de7c1c94ee8a92f2a78f03e654f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 02:36:10.448472 9733 raft_consensus.cc:3037] T 9980a009cc164788ab918d402249ab6d P be444de7c1c94ee8a92f2a78f03e654f [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:36:10.451031 9733 raft_consensus.cc:504] T 9980a009cc164788ab918d402249ab6d P be444de7c1c94ee8a92f2a78f03e654f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c7b0dc3dde34ef98a501ea15c294a1b" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 34685 } } peers { permanent_uuid: "be444de7c1c94ee8a92f2a78f03e654f" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 42511 } } peers { permanent_uuid: "ea68384054b64897b6c13998b543bd1b" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 41005 } }
I20260812 02:36:10.451831 9733 leader_election.cc:288] T 9980a009cc164788ab918d402249ab6d P be444de7c1c94ee8a92f2a78f03e654f [CANDIDATE]: Term 1 election: Requested vote from peers 3c7b0dc3dde34ef98a501ea15c294a1b (127.7.132.129:34685), ea68384054b64897b6c13998b543bd1b (127.7.132.131:41005)
I20260812 02:36:10.452319 9634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9980a009cc164788ab918d402249ab6d" candidate_uuid: "be444de7c1c94ee8a92f2a78f03e654f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea68384054b64897b6c13998b543bd1b"
I20260812 02:36:10.452315 9296 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9980a009cc164788ab918d402249ab6d" candidate_uuid: "be444de7c1c94ee8a92f2a78f03e654f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c7b0dc3dde34ef98a501ea15c294a1b"
I20260812 02:36:10.452569 9634 raft_consensus.cc:3037] T 9980a009cc164788ab918d402249ab6d P ea68384054b64897b6c13998b543bd1b [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:36:10.452590 9296 raft_consensus.cc:3037] T 9980a009cc164788ab918d402249ab6d P 3c7b0dc3dde34ef98a501ea15c294a1b [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:36:10.454705 9634 raft_consensus.cc:2454] T 9980a009cc164788ab918d402249ab6d P ea68384054b64897b6c13998b543bd1b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be444de7c1c94ee8a92f2a78f03e654f in term 1.
I20260812 02:36:10.455161 9374 leader_election.cc:302] T 9980a009cc164788ab918d402249ab6d P be444de7c1c94ee8a92f2a78f03e654f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be444de7c1c94ee8a92f2a78f03e654f, ea68384054b64897b6c13998b543bd1b; no voters:
I20260812 02:36:10.455446 9733 raft_consensus.cc:2781] T 9980a009cc164788ab918d402249ab6d P be444de7c1c94ee8a92f2a78f03e654f [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:36:10.455680 9733 raft_consensus.cc:686] T 9980a009cc164788ab918d402249ab6d P be444de7c1c94ee8a92f2a78f03e654f [term 1 LEADER]: Becoming Leader. State: Replica: be444de7c1c94ee8a92f2a78f03e654f, State: Running, Role: LEADER
I20260812 02:36:10.455839 9296 raft_consensus.cc:2454] T 9980a009cc164788ab918d402249ab6d P 3c7b0dc3dde34ef98a501ea15c294a1b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be444de7c1c94ee8a92f2a78f03e654f in term 1.
I20260812 02:36:10.455966 9733 consensus_queue.cc:237] T 9980a009cc164788ab918d402249ab6d P be444de7c1c94ee8a92f2a78f03e654f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c7b0dc3dde34ef98a501ea15c294a1b" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 34685 } } peers { permanent_uuid: "be444de7c1c94ee8a92f2a78f03e654f" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 42511 } } peers { permanent_uuid: "ea68384054b64897b6c13998b543bd1b" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 41005 } }
I20260812 02:36:10.457954 9740 tablet_replica.cc:439] T 9980a009cc164788ab918d402249ab6d P be444de7c1c94ee8a92f2a78f03e654f: TxnStatusTablet state changed. Reason: New leader be444de7c1c94ee8a92f2a78f03e654f. Latest consensus state: current_term: 1 leader_uuid: "be444de7c1c94ee8a92f2a78f03e654f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c7b0dc3dde34ef98a501ea15c294a1b" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 34685 } } peers { permanent_uuid: "be444de7c1c94ee8a92f2a78f03e654f" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 42511 } } peers { permanent_uuid: "ea68384054b64897b6c13998b543bd1b" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 41005 } } }
I20260812 02:36:10.458384 9740 tablet_replica.cc:442] T 9980a009cc164788ab918d402249ab6d P be444de7c1c94ee8a92f2a78f03e654f: This TxnStatusTablet replica's current role is: LEADER
I20260812 02:36:10.459446 9095 catalog_manager.cc:5495] T 9980a009cc164788ab918d402249ab6d P be444de7c1c94ee8a92f2a78f03e654f reported cstate change: term changed from 0 to 1, leader changed from <none> to be444de7c1c94ee8a92f2a78f03e654f (127.7.132.130). New cstate: current_term: 1 leader_uuid: "be444de7c1c94ee8a92f2a78f03e654f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c7b0dc3dde34ef98a501ea15c294a1b" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 34685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be444de7c1c94ee8a92f2a78f03e654f" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 42511 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ea68384054b64897b6c13998b543bd1b" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 41005 } health_report { overall_health: UNKNOWN } } }
I20260812 02:36:10.463590 9779 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260812 02:36:10.463903 9779 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260812 02:36:10.481446 9723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:10.475540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49640 (local address 127.7.132.130:42511)
0812 02:36:10.475781 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:10.475786 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:10.475807 (+ 21us) server_negotiation.cc:407] Connection header received
0812 02:36:10.476106 (+ 299us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:10.476110 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:10.476182 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:10.476286 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 02:36:10.476743 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:10.477652 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:10.479693 (+ 2041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:10.480145 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:10.480249 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:10.480815 (+ 566us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 02:36:10.480924 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 02:36:10.480997 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:10.481171 (+ 174us) server_negotiation.cc:299] Negotiation successful
0812 02:36:10.481233 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260812 02:36:10.481652 9773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:10.475482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.130:42511 (local address 127.0.0.1:49640)
0812 02:36:10.475734 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:10.475758 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:10.475905 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:10.476335 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:10.476342 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:10.476357 (+ 15us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 02:36:10.476586 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:10.476597 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:10.477795 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:10.477801 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:10.479517 (+ 1716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:10.479535 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:10.480323 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:10.480330 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:10.480565 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:10.480638 (+ 73us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 02:36:10.481031 (+ 393us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 02:36:10.481056 (+ 25us) client_negotiation.cc:769] Sending connection context
0812 02:36:10.481145 (+ 89us) client_negotiation.cc:240] Negotiation successful
0812 02:36:10.481313 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260812 02:36:10.485803 9296 raft_consensus.cc:1261] T 9980a009cc164788ab918d402249ab6d P 3c7b0dc3dde34ef98a501ea15c294a1b [term 1 FOLLOWER]: Refusing update from remote peer be444de7c1c94ee8a92f2a78f03e654f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:36:10.485893 9634 raft_consensus.cc:1261] T 9980a009cc164788ab918d402249ab6d P ea68384054b64897b6c13998b543bd1b [term 1 FOLLOWER]: Refusing update from remote peer be444de7c1c94ee8a92f2a78f03e654f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:36:10.486568 9740 consensus_queue.cc:1035] T 9980a009cc164788ab918d402249ab6d P be444de7c1c94ee8a92f2a78f03e654f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ea68384054b64897b6c13998b543bd1b" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 41005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:36:10.486949 9733 consensus_queue.cc:1035] T 9980a009cc164788ab918d402249ab6d P be444de7c1c94ee8a92f2a78f03e654f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c7b0dc3dde34ef98a501ea15c294a1b" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 34685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:36:10.490784 9734 tablet_replica.cc:439] T 9980a009cc164788ab918d402249ab6d P 3c7b0dc3dde34ef98a501ea15c294a1b: TxnStatusTablet state changed. Reason: New leader be444de7c1c94ee8a92f2a78f03e654f. Latest consensus state: current_term: 1 leader_uuid: "be444de7c1c94ee8a92f2a78f03e654f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c7b0dc3dde34ef98a501ea15c294a1b" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 34685 } } peers { permanent_uuid: "be444de7c1c94ee8a92f2a78f03e654f" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 42511 } } peers { permanent_uuid: "ea68384054b64897b6c13998b543bd1b" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 41005 } } }
I20260812 02:36:10.491245 9734 tablet_replica.cc:442] T 9980a009cc164788ab918d402249ab6d P 3c7b0dc3dde34ef98a501ea15c294a1b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 02:36:10.496868 9734 tablet_replica.cc:439] T 9980a009cc164788ab918d402249ab6d P 3c7b0dc3dde34ef98a501ea15c294a1b: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "be444de7c1c94ee8a92f2a78f03e654f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c7b0dc3dde34ef98a501ea15c294a1b" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 34685 } } peers { permanent_uuid: "be444de7c1c94ee8a92f2a78f03e654f" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 42511 } } peers { permanent_uuid: "ea68384054b64897b6c13998b543bd1b" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 41005 } } }
I20260812 02:36:10.497051 9745 mvcc.cc:205] Tried to move back new op lower bound from 7317512890303385600 to 7317512890190180352. Current Snapshot: MvccSnapshot[applied={T|T < 7317512890303385600}]
I20260812 02:36:10.497210 9734 tablet_replica.cc:442] T 9980a009cc164788ab918d402249ab6d P 3c7b0dc3dde34ef98a501ea15c294a1b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 02:36:10.496565 9740 tablet_replica.cc:439] T 9980a009cc164788ab918d402249ab6d P be444de7c1c94ee8a92f2a78f03e654f: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "be444de7c1c94ee8a92f2a78f03e654f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c7b0dc3dde34ef98a501ea15c294a1b" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 34685 } } peers { permanent_uuid: "be444de7c1c94ee8a92f2a78f03e654f" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 42511 } } peers { permanent_uuid: "ea68384054b64897b6c13998b543bd1b" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 41005 } } }
I20260812 02:36:10.499531 9740 tablet_replica.cc:442] T 9980a009cc164788ab918d402249ab6d P be444de7c1c94ee8a92f2a78f03e654f: This TxnStatusTablet replica's current role is: LEADER
I20260812 02:36:10.501137 9735 tablet_replica.cc:439] T 9980a009cc164788ab918d402249ab6d P ea68384054b64897b6c13998b543bd1b: TxnStatusTablet state changed. Reason: New leader be444de7c1c94ee8a92f2a78f03e654f. Latest consensus state: current_term: 1 leader_uuid: "be444de7c1c94ee8a92f2a78f03e654f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c7b0dc3dde34ef98a501ea15c294a1b" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 34685 } } peers { permanent_uuid: "be444de7c1c94ee8a92f2a78f03e654f" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 42511 } } peers { permanent_uuid: "ea68384054b64897b6c13998b543bd1b" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 41005 } } }
I20260812 02:36:10.501436 9735 tablet_replica.cc:442] T 9980a009cc164788ab918d402249ab6d P ea68384054b64897b6c13998b543bd1b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 02:36:10.502720 9784 tablet_replica.cc:439] T 9980a009cc164788ab918d402249ab6d P be444de7c1c94ee8a92f2a78f03e654f: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "be444de7c1c94ee8a92f2a78f03e654f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c7b0dc3dde34ef98a501ea15c294a1b" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 34685 } } peers { permanent_uuid: "be444de7c1c94ee8a92f2a78f03e654f" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 42511 } } peers { permanent_uuid: "ea68384054b64897b6c13998b543bd1b" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 41005 } } }
I20260812 02:36:10.503173 9784 tablet_replica.cc:442] T 9980a009cc164788ab918d402249ab6d P be444de7c1c94ee8a92f2a78f03e654f: This TxnStatusTablet replica's current role is: LEADER
I20260812 02:36:10.507375 9735 tablet_replica.cc:439] T 9980a009cc164788ab918d402249ab6d P ea68384054b64897b6c13998b543bd1b: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "be444de7c1c94ee8a92f2a78f03e654f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c7b0dc3dde34ef98a501ea15c294a1b" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 34685 } } peers { permanent_uuid: "be444de7c1c94ee8a92f2a78f03e654f" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 42511 } } peers { permanent_uuid: "ea68384054b64897b6c13998b543bd1b" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 41005 } } }
I20260812 02:36:10.507647 9735 tablet_replica.cc:442] T 9980a009cc164788ab918d402249ab6d P ea68384054b64897b6c13998b543bd1b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 02:36:10.517336 9721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:10.510995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55012 (local address 127.7.132.129:34685)
0812 02:36:10.511184 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:10.511189 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:10.511284 (+ 95us) server_negotiation.cc:407] Connection header received
0812 02:36:10.511515 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:10.511519 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:10.511596 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:10.511721 (+ 125us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 02:36:10.512153 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:10.512891 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:10.514907 (+ 2016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:10.515297 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:10.515392 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:10.515958 (+ 566us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 02:36:10.516570 (+ 612us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 02:36:10.516741 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:10.516895 (+ 154us) server_negotiation.cc:299] Negotiation successful
0812 02:36:10.516971 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260812 02:36:10.534415 9723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:10.528377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:35933 (local address 127.7.132.130:42511)
0812 02:36:10.528580 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:10.528584 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:10.529181 (+ 597us) server_negotiation.cc:407] Connection header received
0812 02:36:10.529398 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:10.529403 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:10.529482 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:10.529581 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 02:36:10.530074 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:10.530882 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:10.532524 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:10.533001 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:10.533145 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:10.533599 (+ 454us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 02:36:10.533716 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 02:36:10.533825 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:10.533968 (+ 143us) server_negotiation.cc:299] Negotiation successful
0812 02:36:10.534049 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 02:36:10.534489 9788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:10.528471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.130:42511 (local address 127.7.132.129:35933)
0812 02:36:10.529073 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:10.529089 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:10.529257 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:10.529594 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:10.529598 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:10.529609 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 02:36:10.529907 (+ 298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:10.529916 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:10.531020 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:10.531028 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:10.532353 (+ 1325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:10.532365 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:10.533187 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:10.533193 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:10.533345 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:10.533397 (+ 52us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 02:36:10.533823 (+ 426us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 02:36:10.533836 (+ 13us) client_negotiation.cc:769] Sending connection context
0812 02:36:10.533944 (+ 108us) client_negotiation.cc:240] Negotiation successful
0812 02:36:10.534102 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":314,"threads_started":1}
I20260812 02:36:10.545410 9634 raft_consensus.cc:1261] T 5d917e1b434a4b4aa6cfd02f0921ff63 P ea68384054b64897b6c13998b543bd1b [term 1 FOLLOWER]: Refusing update from remote peer 3c7b0dc3dde34ef98a501ea15c294a1b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:36:10.545779 9461 raft_consensus.cc:1261] T 5d917e1b434a4b4aa6cfd02f0921ff63 P be444de7c1c94ee8a92f2a78f03e654f [term 1 FOLLOWER]: Refusing update from remote peer 3c7b0dc3dde34ef98a501ea15c294a1b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:36:10.546113 9734 consensus_queue.cc:1035] T 5d917e1b434a4b4aa6cfd02f0921ff63 P 3c7b0dc3dde34ef98a501ea15c294a1b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ea68384054b64897b6c13998b543bd1b" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 41005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:36:10.546653 9761 consensus_queue.cc:1035] T 5d917e1b434a4b4aa6cfd02f0921ff63 P 3c7b0dc3dde34ef98a501ea15c294a1b [LEADER]: Connected to new peer: Peer: permanent_uuid: "be444de7c1c94ee8a92f2a78f03e654f" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 42511 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:36:10.553872 9744 mvcc.cc:205] Tried to move back new op lower bound from 7317512890546745344 to 7317512889602355200. Current Snapshot: MvccSnapshot[applied={T|T < 7317512890546745344}]
I20260812 02:36:10.588006 9721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:10.580389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:60685 (local address 127.7.132.129:34685)
0812 02:36:10.580662 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:10.580670 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:10.581025 (+ 355us) server_negotiation.cc:407] Connection header received
0812 02:36:10.581319 (+ 294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:10.581325 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:10.581416 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:10.581530 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 02:36:10.582035 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:10.583187 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:10.585812 (+ 2625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:10.586416 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:10.586527 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:10.587175 (+ 648us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 02:36:10.587314 (+ 139us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 02:36:10.587439 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:10.587600 (+ 161us) server_negotiation.cc:299] Negotiation successful
0812 02:36:10.587679 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108}
I20260812 02:36:10.588119 9792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:10.579941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:34685 (local address 127.7.132.130:60685)
0812 02:36:10.580922 (+ 981us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:10.580941 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:10.581154 (+ 213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:10.581549 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:10.581552 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:10.581562 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 02:36:10.581850 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:10.581859 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:10.583355 (+ 1496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:10.583361 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:10.585661 (+ 2300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:10.585677 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:10.586632 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:10.586640 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:10.586827 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:10.586910 (+ 83us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 02:36:10.587441 (+ 531us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 02:36:10.587465 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 02:36:10.587576 (+ 111us) client_negotiation.cc:240] Negotiation successful
0812 02:36:10.587710 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":775,"thread_start_us":605,"threads_started":1}
I20260812 02:36:10.632141 9095 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50278:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 02:36:10.632417 9095 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50278:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 02:36:10.636497 9095 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 38e2dd3af22545f6b021313a489f7387: Sending DeleteTablet for 3 replicas of tablet 5d917e1b434a4b4aa6cfd02f0921ff63
I20260812 02:36:10.637616 9428 tablet_service.cc:1514] Processing DeleteTablet for tablet 5d917e1b434a4b4aa6cfd02f0921ff63 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:36:10 UTC) from {username='kudu', principal='kudu/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:49636
I20260812 02:36:10.637636 9607 tablet_service.cc:1514] Processing DeleteTablet for tablet 5d917e1b434a4b4aa6cfd02f0921ff63 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:36:10 UTC) from {username='kudu', principal='kudu/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:54050
I20260812 02:36:10.637951 9754 tablet_replica.cc:330] T 5d917e1b434a4b4aa6cfd02f0921ff63 P be444de7c1c94ee8a92f2a78f03e654f: stopping tablet replica
I20260812 02:36:10.637938 9755 tablet_replica.cc:330] T 5d917e1b434a4b4aa6cfd02f0921ff63 P ea68384054b64897b6c13998b543bd1b: stopping tablet replica
I20260812 02:36:10.638190 9754 raft_consensus.cc:2229] T 5d917e1b434a4b4aa6cfd02f0921ff63 P be444de7c1c94ee8a92f2a78f03e654f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 02:36:10.638183 9755 raft_consensus.cc:2229] T 5d917e1b434a4b4aa6cfd02f0921ff63 P ea68384054b64897b6c13998b543bd1b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 02:36:10.638389 9755 raft_consensus.cc:2258] T 5d917e1b434a4b4aa6cfd02f0921ff63 P ea68384054b64897b6c13998b543bd1b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:36:10.638396 9754 raft_consensus.cc:2258] T 5d917e1b434a4b4aa6cfd02f0921ff63 P be444de7c1c94ee8a92f2a78f03e654f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:36:10.639258 9263 tablet_service.cc:1514] Processing DeleteTablet for tablet 5d917e1b434a4b4aa6cfd02f0921ff63 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:36:10 UTC) from {username='kudu', principal='kudu/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:55008
I20260812 02:36:10.639593 9752 tablet_replica.cc:330] T 5d917e1b434a4b4aa6cfd02f0921ff63 P 3c7b0dc3dde34ef98a501ea15c294a1b: stopping tablet replica
I20260812 02:36:10.639804 9752 raft_consensus.cc:2229] T 5d917e1b434a4b4aa6cfd02f0921ff63 P 3c7b0dc3dde34ef98a501ea15c294a1b [term 1 LEADER]: Raft consensus shutting down.
I20260812 02:36:10.640376 9752 raft_consensus.cc:2258] T 5d917e1b434a4b4aa6cfd02f0921ff63 P 3c7b0dc3dde34ef98a501ea15c294a1b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:36:10.641263 9754 ts_tablet_manager.cc:1905] T 5d917e1b434a4b4aa6cfd02f0921ff63 P be444de7c1c94ee8a92f2a78f03e654f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:36:10.641866 9755 ts_tablet_manager.cc:1905] T 5d917e1b434a4b4aa6cfd02f0921ff63 P ea68384054b64897b6c13998b543bd1b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:36:10.647248 9752 ts_tablet_manager.cc:1905] T 5d917e1b434a4b4aa6cfd02f0921ff63 P 3c7b0dc3dde34ef98a501ea15c294a1b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:36:10.656646 9752 ts_tablet_manager.cc:1918] T 5d917e1b434a4b4aa6cfd02f0921ff63 P 3c7b0dc3dde34ef98a501ea15c294a1b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 02:36:10.656842 9752 log.cc:1196] T 5d917e1b434a4b4aa6cfd02f0921ff63 P 3c7b0dc3dde34ef98a501ea15c294a1b: Deleting WAL directory at /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502156971023-7698-0/minicluster-data/ts-0/wal/wals/5d917e1b434a4b4aa6cfd02f0921ff63
I20260812 02:36:10.657338 9752 ts_tablet_manager.cc:1939] T 5d917e1b434a4b4aa6cfd02f0921ff63 P 3c7b0dc3dde34ef98a501ea15c294a1b: Deleting consensus metadata
I20260812 02:36:10.658746 9079 catalog_manager.cc:4841] TS 3c7b0dc3dde34ef98a501ea15c294a1b (127.7.132.129:34685): tablet 5d917e1b434a4b4aa6cfd02f0921ff63 (table test-table [id=8f47d4e40a6b45fbab781ddbb30b0d72]) successfully deleted
I20260812 02:36:10.661368 9755 ts_tablet_manager.cc:1918] T 5d917e1b434a4b4aa6cfd02f0921ff63 P ea68384054b64897b6c13998b543bd1b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 02:36:10.661542 9755 log.cc:1196] T 5d917e1b434a4b4aa6cfd02f0921ff63 P ea68384054b64897b6c13998b543bd1b: Deleting WAL directory at /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502156971023-7698-0/minicluster-data/ts-2/wal/wals/5d917e1b434a4b4aa6cfd02f0921ff63
I20260812 02:36:10.662022 9755 ts_tablet_manager.cc:1939] T 5d917e1b434a4b4aa6cfd02f0921ff63 P ea68384054b64897b6c13998b543bd1b: Deleting consensus metadata
I20260812 02:36:10.663295 9077 catalog_manager.cc:4841] TS ea68384054b64897b6c13998b543bd1b (127.7.132.131:41005): tablet 5d917e1b434a4b4aa6cfd02f0921ff63 (table test-table [id=8f47d4e40a6b45fbab781ddbb30b0d72]) successfully deleted
I20260812 02:36:10.670360 9754 ts_tablet_manager.cc:1918] T 5d917e1b434a4b4aa6cfd02f0921ff63 P be444de7c1c94ee8a92f2a78f03e654f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 02:36:10.670624 9754 log.cc:1196] T 5d917e1b434a4b4aa6cfd02f0921ff63 P be444de7c1c94ee8a92f2a78f03e654f: Deleting WAL directory at /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786502156971023-7698-0/minicluster-data/ts-1/wal/wals/5d917e1b434a4b4aa6cfd02f0921ff63
I20260812 02:36:10.671154 9754 ts_tablet_manager.cc:1939] T 5d917e1b434a4b4aa6cfd02f0921ff63 P be444de7c1c94ee8a92f2a78f03e654f: Deleting consensus metadata
I20260812 02:36:10.672595 9075 catalog_manager.cc:4841] TS be444de7c1c94ee8a92f2a78f03e654f (127.7.132.130:42511): tablet 5d917e1b434a4b4aa6cfd02f0921ff63 (table test-table [id=8f47d4e40a6b45fbab781ddbb30b0d72]) successfully deleted
Aug 12 02:36:10 dist-test-slave-mn4r krb5kdc[9040](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502169, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.132.129@KRBTEST.COM
I20260812 02:36:10.705597 9721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:10.673697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55020 (local address 127.7.132.129:34685)
0812 02:36:10.673919 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:10.673925 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:10.685175 (+ 11250us) server_negotiation.cc:407] Connection header received
0812 02:36:10.685597 (+ 422us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:10.685603 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:10.685709 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:10.685787 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:10.688081 (+ 2294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:10.689490 (+ 1409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:10.691451 (+ 1961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:10.692157 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:10.692257 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:10.697770 (+ 5513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:10.697784 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:10.697807 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:10.697852 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:10.702018 (+ 4166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:10.702959 (+ 941us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:10.702969 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:10.702972 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:10.703136 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:10.703551 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:10.703556 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:10.703559 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:10.704037 (+ 478us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:10.704383 (+ 346us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:10.704951 (+ 568us) server_negotiation.cc:299] Negotiation successful
0812 02:36:10.705195 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
W20260812 02:36:10.707226 9343 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55020
I20260812 02:36:10.728902 9770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:10.712142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50302 (local address 127.7.132.190:43261)
0812 02:36:10.712327 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:10.712334 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:10.712794 (+ 460us) server_negotiation.cc:407] Connection header received
0812 02:36:10.713069 (+ 275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:10.713076 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:10.713171 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:10.713287 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:10.714950 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:10.716118 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:10.718158 (+ 2040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:10.718831 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:10.718968 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:10.722211 (+ 3243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:10.722218 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:10.722223 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:10.722265 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:10.725863 (+ 3598us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:10.726676 (+ 813us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:10.726682 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:10.726685 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:10.726828 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:10.727264 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:10.727270 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:10.727273 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:10.727650 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:10.727892 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:10.728303 (+ 411us) server_negotiation.cc:299] Negotiation successful
0812 02:36:10.728547 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
W20260812 02:36:10.729550 9095 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50302
I20260812 02:36:10.732036 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 9187
I20260812 02:36:10.751770 9721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:10.734839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55022 (local address 127.7.132.129:34685)
0812 02:36:10.735010 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:10.735016 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:10.735035 (+ 19us) server_negotiation.cc:407] Connection header received
0812 02:36:10.735314 (+ 279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:10.735326 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:10.735413 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:10.735517 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:10.737191 (+ 1674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:10.738395 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:10.740281 (+ 1886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:10.740933 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:10.741063 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:10.744147 (+ 3084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:10.744154 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:10.744159 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:10.744208 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:10.748087 (+ 3879us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:10.749135 (+ 1048us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:10.749142 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:10.749146 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:10.749332 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:10.749788 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:10.749794 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:10.749798 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:10.750246 (+ 448us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:10.750507 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:10.750940 (+ 433us) server_negotiation.cc:299] Negotiation successful
0812 02:36:10.751217 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
W20260812 02:36:10.752493 9343 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55022
W20260812 02:36:10.753202 7698 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 9187: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 02:36:10.753386 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 9187
I20260812 02:36:10.785118 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 9355
Aug 12 02:36:10 dist-test-slave-mn4r krb5kdc[9040](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502169, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.132.130@KRBTEST.COM
I20260812 02:36:10.805163 9723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:10.785912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49648 (local address 127.7.132.130:42511)
0812 02:36:10.786140 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:10.786146 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:10.786391 (+ 245us) server_negotiation.cc:407] Connection header received
0812 02:36:10.786622 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:10.786628 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:10.786726 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:10.786834 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:10.788453 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:10.789617 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:10.791640 (+ 2023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:10.792333 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:10.792459 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:10.797838 (+ 5379us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:10.797858 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:10.797876 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:10.797942 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:10.802097 (+ 4155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:10.802866 (+ 769us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:10.802876 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:10.802880 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:10.803054 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:10.803510 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:10.803516 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:10.803520 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:10.804029 (+ 509us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:10.804346 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:10.804536 (+ 190us) server_negotiation.cc:299] Negotiation successful
0812 02:36:10.804770 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
W20260812 02:36:10.805943 9514 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49648
W20260812 02:36:10.806648 7698 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 9355: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 02:36:10.806831 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 9355
I20260812 02:36:10.835848 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 9525
Aug 12 02:36:10 dist-test-slave-mn4r krb5kdc[9040](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502169, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.132.131@KRBTEST.COM
I20260812 02:36:10.851934 9804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:10.836626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54064 (local address 127.7.132.131:41005)
0812 02:36:10.837127 (+ 501us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:10.837131 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:10.837149 (+ 18us) server_negotiation.cc:407] Connection header received
0812 02:36:10.837207 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:10.837211 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:10.837272 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:10.837373 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:10.838772 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:10.839640 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:10.840967 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:10.841403 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:10.841539 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:10.845851 (+ 4312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:10.845866 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:10.845883 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:10.845920 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:10.849170 (+ 3250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:10.849821 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:10.849829 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:10.849831 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:10.849969 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:10.850368 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:10.850372 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:10.850374 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:10.850837 (+ 463us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:10.851132 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:10.851379 (+ 247us) server_negotiation.cc:299] Negotiation successful
0812 02:36:10.851623 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":408,"thread_start_us":274,"threads_started":1}
W20260812 02:36:10.852788 9690 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54064
W20260812 02:36:10.853483 7698 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 9525: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 02:36:10.853623 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 9525
I20260812 02:36:10.881388 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 9056
I20260812 02:36:10.882678 9149 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:36:10.999264 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 9056
2026-08-12T02:36:11Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (4834 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502156971023-7698-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 02:36:11 dist-test-slave-mn4r krb5kdc[9811](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 02:36:11 dist-test-slave-mn4r krb5kdc[9811](info): set up 2 sockets
Aug 12 02:36:11 dist-test-slave-mn4r krb5kdc[9811](info): commencing operation
krb5kdc: starting...
W20260812 02:36:12.248564 7698 mini_kdc.cc:121] Time spent starting KDC: real 1.200s user 0.001s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502156971023-7698-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502156971023-7698-0/krb5kdc/test-user.keytab.
Aug 12 02:36:12 dist-test-slave-mn4r krb5kdc[9811](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502172, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T02:36:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T02:36:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.132.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
I20260812 02:36:12.466109 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502156971023-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502156971023-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502156971023-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:39693
--webserver_interface=127.7.132.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:34815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.132.190:39693
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:36:12.801038 9829 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:36:12.801347 9829 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:36:12.801529 9829 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:36:12.808866 9829 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:36:12.808986 9829 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260812 02:36:12.809082 9829 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:36:12.809203 9829 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:36:12.809288 9829 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:36:12.809404 9829 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:36:12.819440 9829 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:34815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502156971023-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502156971023-7698-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.132.190:39693
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:39693
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.132.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502156971023-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:36:12.820740 9829 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:36:12.822180 9829 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:36:12.834620 9835 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:36:12.834549 9836 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:36:12.834596 9838 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:36:12.835557 9829 server_base.cc:1034] running on GCE node
I20260812 02:36:12.836364 9829 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:36:12.837807 9829 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:36:12.839068 9829 hybrid_clock.cc:648] HybridClock initialized: now 1786502172838990 us; error 40 us; skew 500 ppm
I20260812 02:36:12.839449 9829 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:36:12 dist-test-slave-mn4r krb5kdc[9811](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502172, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:36:12.843212 9829 init.cc:377] Logged in from keytab as kudu/127.7.132.190@KRBTEST.COM (short username kudu)
I20260812 02:36:12.844852 9829 webserver.cc:460] Webserver started at http://127.7.132.190:36363/ using document root <none> and password file <none>
I20260812 02:36:12.845624 9829 fs_manager.cc:359] Metadata directory not provided
I20260812 02:36:12.845762 9829 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:36:12.846172 9829 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:36:12.849205 9829 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502156971023-7698-0/minicluster-data/master-0/data/instance:
uuid: "1a4320c66f614d04a17f19477a85c319"
format_stamp: "Formatted at 2026-08-12 02:36:12 on dist-test-slave-mn4r"
server_key: "9cc2aa38587fea01f095520dbff37a9c"
server_key_iv: "49c03109e91ecbeca8c363084fc6dd07"
server_key_version: "encryptionkey@0"
I20260812 02:36:12.849987 9829 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502156971023-7698-0/minicluster-data/master-0/wal/instance:
uuid: "1a4320c66f614d04a17f19477a85c319"
format_stamp: "Formatted at 2026-08-12 02:36:12 on dist-test-slave-mn4r"
server_key: "9cc2aa38587fea01f095520dbff37a9c"
server_key_iv: "49c03109e91ecbeca8c363084fc6dd07"
server_key_version: "encryptionkey@0"
I20260812 02:36:12.855345 9829 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260812 02:36:12.858954 9847 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:36:12.860368 9829 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 02:36:12.860574 9829 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502156971023-7698-0/minicluster-data/master-0/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502156971023-7698-0/minicluster-data/master-0/wal
uuid: "1a4320c66f614d04a17f19477a85c319"
format_stamp: "Formatted at 2026-08-12 02:36:12 on dist-test-slave-mn4r"
server_key: "9cc2aa38587fea01f095520dbff37a9c"
server_key_iv: "49c03109e91ecbeca8c363084fc6dd07"
server_key_version: "encryptionkey@0"
I20260812 02:36:12.860811 9829 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502156971023-7698-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502156971023-7698-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502156971023-7698-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:36:12.875614 9829 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:36:12.880353 9829 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:36:12.880647 9829 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:36:12.903841 9829 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.190:39693
I20260812 02:36:12.903863 9920 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.190:39693 every 8 connection(s)
I20260812 02:36:12.905339 9829 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
I20260812 02:36:12.907043 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 9829
I20260812 02:36:12.907276 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502156971023-7698-0/minicluster-data/master-0/wal/instance
I20260812 02:36:12.908144 7698 external_mini_cluster.cc:1428] Setting key b6e880127255c02bdabf782795d950b6
I20260812 02:36:12.910554 9921 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 12 02:36:12 dist-test-slave-mn4r krb5kdc[9811](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502172, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 02:36:12.924414 9921 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a4320c66f614d04a17f19477a85c319: Bootstrap starting.
I20260812 02:36:12.927772 9921 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1a4320c66f614d04a17f19477a85c319: Neither blocks nor log segments found. Creating new log.
I20260812 02:36:12.928894 9921 log.cc:824] T 00000000000000000000000000000000 P 1a4320c66f614d04a17f19477a85c319: Log is configured to *not* fsync() on all Append() calls
I20260812 02:36:12.932413 9921 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a4320c66f614d04a17f19477a85c319: No bootstrap required, opened a new log
I20260812 02:36:12.932772 9925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:12.910414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49402 (local address 127.7.132.190:39693)
0812 02:36:12.911316 (+ 902us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:12.911328 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:12.911371 (+ 43us) server_negotiation.cc:407] Connection header received
0812 02:36:12.913037 (+ 1666us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:12.913061 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:12.913505 (+ 444us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:12.913862 (+ 357us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:12.915774 (+ 1912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:12.917214 (+ 1440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:12.918969 (+ 1755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:12.919662 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:12.919801 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:12.924708 (+ 4907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:12.924717 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:12.924732 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:12.924771 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:12.928018 (+ 3247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:12.928800 (+ 782us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:12.928814 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:12.928822 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:12.929005 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:12.929561 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:12.929568 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:12.929573 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:12.930463 (+ 890us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:12.930692 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:12.931241 (+ 549us) server_negotiation.cc:299] Negotiation successful
0812 02:36:12.931640 (+ 399us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":678,"thread_start_us":460,"threads_started":1}
I20260812 02:36:12.940064 9921 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1a4320c66f614d04a17f19477a85c319 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a4320c66f614d04a17f19477a85c319" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 39693 } }
I20260812 02:36:12.940445 9921 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1a4320c66f614d04a17f19477a85c319 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:36:12.940557 9921 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1a4320c66f614d04a17f19477a85c319 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a4320c66f614d04a17f19477a85c319, State: Initialized, Role: FOLLOWER
I20260812 02:36:12.941108 9921 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1a4320c66f614d04a17f19477a85c319 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a4320c66f614d04a17f19477a85c319" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 39693 } }
I20260812 02:36:12.941365 9921 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1a4320c66f614d04a17f19477a85c319 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 02:36:12.941522 9921 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1a4320c66f614d04a17f19477a85c319 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 02:36:12.941706 9921 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1a4320c66f614d04a17f19477a85c319 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:36:12.944073 9921 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1a4320c66f614d04a17f19477a85c319 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a4320c66f614d04a17f19477a85c319" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 39693 } }
I20260812 02:36:12.944615 9921 leader_election.cc:302] T 00000000000000000000000000000000 P 1a4320c66f614d04a17f19477a85c319 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1a4320c66f614d04a17f19477a85c319; no voters:
I20260812 02:36:12.945200 9921 leader_election.cc:288] T 00000000000000000000000000000000 P 1a4320c66f614d04a17f19477a85c319 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 02:36:12.945364 9928 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1a4320c66f614d04a17f19477a85c319 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:36:12.946259 9928 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1a4320c66f614d04a17f19477a85c319 [term 1 LEADER]: Becoming Leader. State: Replica: 1a4320c66f614d04a17f19477a85c319, State: Running, Role: LEADER
I20260812 02:36:12.946816 9928 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1a4320c66f614d04a17f19477a85c319 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a4320c66f614d04a17f19477a85c319" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 39693 } }
I20260812 02:36:12.947451 9921 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1a4320c66f614d04a17f19477a85c319 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:36:12.949807 9929 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a4320c66f614d04a17f19477a85c319 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1a4320c66f614d04a17f19477a85c319" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a4320c66f614d04a17f19477a85c319" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 39693 } } }
I20260812 02:36:12.950286 9929 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a4320c66f614d04a17f19477a85c319 [sys.catalog]: This master's current role is: LEADER
I20260812 02:36:12.950781 9930 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a4320c66f614d04a17f19477a85c319 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1a4320c66f614d04a17f19477a85c319. Latest consensus state: current_term: 1 leader_uuid: "1a4320c66f614d04a17f19477a85c319" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a4320c66f614d04a17f19477a85c319" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 39693 } } }
I20260812 02:36:12.951103 9930 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a4320c66f614d04a17f19477a85c319 [sys.catalog]: This master's current role is: LEADER
I20260812 02:36:12.952469 9934 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:36:12.959179 9934 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 02:36:12.969527 9934 catalog_manager.cc:1347] Generated new cluster ID: f6fb116fe1f54daba60041b996227a42
I20260812 02:36:12.969660 9934 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 02:36:12.981585 9934 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 02:36:12.983636 9934 catalog_manager.cc:1504] Loading token signing keys...
I20260812 02:36:12.993520 9934 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1a4320c66f614d04a17f19477a85c319: Generated new TSK 0
I20260812 02:36:12.994688 9934 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.132.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.129.keytab.
Entry for principal kudu/127.7.132.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.129.keytab.
I20260812 02:36:13.045883 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502156971023-7698-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502156971023-7698-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502156971023-7698-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502156971023-7698-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.129:0
--local_ip_for_outbound_sockets=127.7.132.129
--webserver_interface=127.7.132.129
--webserver_port=0
--tserver_master_addrs=127.7.132.190:39693
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.129.keytab
--principal=kudu/127.7.132.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:34815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:36:13.397734 9952 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:36:13.398006 9952 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:36:13.398214 9952 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:36:13.405532 9952 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:36:13.405642 9952 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:36:13.405807 9952 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260812 02:36:13.405886 9952 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 02:36:13.405998 9952 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.129
I20260812 02:36:13.415920 9952 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:34815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502156971023-7698-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502156971023-7698-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.129.keytab
--principal=kudu/127.7.132.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502156971023-7698-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.132.129
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.132.190:39693
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.129
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502156971023-7698-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:36:13.417287 9952 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:36:13.418797 9952 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:36:13.432746 9958 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:36:13.432717 9959 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:36:13.433023 9961 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:36:13.433811 9952 server_base.cc:1034] running on GCE node
I20260812 02:36:13.434589 9952 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:36:13.435590 9952 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:36:13.436795 9952 hybrid_clock.cc:648] HybridClock initialized: now 1786502173436749 us; error 34 us; skew 500 ppm
I20260812 02:36:13.437135 9952 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:36:13 dist-test-slave-mn4r krb5kdc[9811](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502173, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:36:13.440508 9952 init.cc:377] Logged in from keytab as kudu/127.7.132.129@KRBTEST.COM (short username kudu)
I20260812 02:36:13.441992 9952 webserver.cc:460] Webserver started at http://127.7.132.129:40775/ using document root <none> and password file <none>
I20260812 02:36:13.442842 9952 fs_manager.cc:359] Metadata directory not provided
I20260812 02:36:13.442977 9952 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:36:13.443311 9952 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:36:13.446640 9952 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502156971023-7698-0/minicluster-data/ts-0/data/instance:
uuid: "2a89947c31174fd2a5b30285f3c23691"
format_stamp: "Formatted at 2026-08-12 02:36:13 on dist-test-slave-mn4r"
server_key: "e24334fb73c97edb0f32c3f13d9aef88"
server_key_iv: "a8fe78864b8b22004923cc73a590e88d"
server_key_version: "encryptionkey@0"
I20260812 02:36:13.447433 9952 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502156971023-7698-0/minicluster-data/ts-0/wal/instance:
uuid: "2a89947c31174fd2a5b30285f3c23691"
format_stamp: "Formatted at 2026-08-12 02:36:13 on dist-test-slave-mn4r"
server_key: "e24334fb73c97edb0f32c3f13d9aef88"
server_key_iv: "a8fe78864b8b22004923cc73a590e88d"
server_key_version: "encryptionkey@0"
I20260812 02:36:13.453002 9952 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260812 02:36:13.456707 9970 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:36:13.458007 9952 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 02:36:13.458225 9952 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502156971023-7698-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502156971023-7698-0/minicluster-data/ts-0/wal
uuid: "2a89947c31174fd2a5b30285f3c23691"
format_stamp: "Formatted at 2026-08-12 02:36:13 on dist-test-slave-mn4r"
server_key: "e24334fb73c97edb0f32c3f13d9aef88"
server_key_iv: "a8fe78864b8b22004923cc73a590e88d"
server_key_version: "encryptionkey@0"
I20260812 02:36:13.458423 9952 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502156971023-7698-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502156971023-7698-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502156971023-7698-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:36:13.472957 9952 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:36:13.477362 9952 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:36:13.477649 9952 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:36:13.480297 9952 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:36:13.480437 9952 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:36:13.480566 9952 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:36:13.480659 9952 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 02:36:13 dist-test-slave-mn4r krb5kdc[9811](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502173, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.129@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 02:36:13.506675 9982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:13.481111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:43801 (local address 127.7.132.190:39693)
0812 02:36:13.481623 (+ 512us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:13.481630 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:13.482881 (+ 1251us) server_negotiation.cc:407] Connection header received
0812 02:36:13.485029 (+ 2148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:13.485034 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:13.485135 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:13.485256 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:13.487831 (+ 2575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:13.488798 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:13.492204 (+ 3406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:13.492829 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:13.492940 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:13.498819 (+ 5879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:13.498825 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:13.498829 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:13.498866 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:13.501768 (+ 2902us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:13.503094 (+ 1326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:13.503102 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:13.503108 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:13.503273 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:13.503955 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:13.503961 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:13.503964 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:13.504396 (+ 432us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:13.504572 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:13.505759 (+ 1187us) server_negotiation.cc:299] Negotiation successful
0812 02:36:13.506150 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":385,"thread_start_us":246,"threads_started":1}
I20260812 02:36:13.507948 9984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:13.481736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:39693 (local address 127.7.132.129:43801)
0812 02:36:13.482690 (+ 954us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:13.482746 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:13.484713 (+ 1967us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:13.485711 (+ 998us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:13.485720 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:13.486751 (+ 1031us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:36:13.487629 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:13.487651 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:13.489029 (+ 1378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:13.489056 (+ 27us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:13.491998 (+ 2942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:13.492017 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:13.493126 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:13.493136 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:13.493523 (+ 387us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:13.494587 (+ 1064us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:36:13.494626 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:36:13.498473 (+ 3847us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:36:13.501944 (+ 3471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:13.501958 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:13.501988 (+ 30us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:13.502913 (+ 925us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:13.503422 (+ 509us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:13.503427 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:13.503431 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:13.503818 (+ 387us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:13.504546 (+ 728us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:36:13.504554 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:36:13.505175 (+ 621us) client_negotiation.cc:769] Sending connection context
0812 02:36:13.505591 (+ 416us) client_negotiation.cc:240] Negotiation successful
0812 02:36:13.506214 (+ 623us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":669,"thread_start_us":435,"threads_started":1}
I20260812 02:36:13.531211 9982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:13.514557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:46697 (local address 127.7.132.190:39693)
0812 02:36:13.514718 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:13.514723 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:13.515468 (+ 745us) server_negotiation.cc:407] Connection header received
0812 02:36:13.515696 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:13.515708 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:13.515819 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:13.515933 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:13.517241 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:13.518564 (+ 1323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:13.519984 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:13.520741 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:13.520901 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:13.523380 (+ 2479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:13.523388 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:13.523397 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:13.523442 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:13.527454 (+ 4012us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:13.528689 (+ 1235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:13.528698 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:13.528704 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:13.528889 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:13.529374 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:13.529382 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:13.529387 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:13.529788 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:13.529935 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:13.530441 (+ 506us) server_negotiation.cc:299] Negotiation successful
0812 02:36:13.530721 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 02:36:13.531348 10072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:13.514695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:39693 (local address 127.7.132.129:46697)
0812 02:36:13.515356 (+ 661us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:13.515372 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:13.515556 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:13.515987 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:13.515995 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:13.516777 (+ 782us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:36:13.517077 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:13.517086 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:13.518727 (+ 1641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:13.518735 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:13.519830 (+ 1095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:13.519843 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:13.520977 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:13.520986 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:13.521174 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:13.522005 (+ 831us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:36:13.522049 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:36:13.523178 (+ 1129us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:36:13.527662 (+ 4484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:13.527670 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:13.527682 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:13.528508 (+ 826us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:13.529027 (+ 519us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:13.529033 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:13.529036 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:13.529225 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:13.529908 (+ 683us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:36:13.529912 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:36:13.530139 (+ 227us) client_negotiation.cc:769] Sending connection context
0812 02:36:13.530286 (+ 147us) client_negotiation.cc:240] Negotiation successful
0812 02:36:13.530752 (+ 466us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":536,"thread_start_us":368,"threads_started":1}
I20260812 02:36:13.532205 9952 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.129:45769
I20260812 02:36:13.532303 10116 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.129:45769 every 8 connection(s)
I20260812 02:36:13.533780 9952 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502156971023-7698-0/minicluster-data/ts-0/data/info.pb
I20260812 02:36:13.536165 10117 heartbeater.cc:347] Connected to a master server at 127.7.132.190:39693
I20260812 02:36:13.536407 10117 heartbeater.cc:464] Registering TS with master...
I20260812 02:36:13.537294 10117 heartbeater.cc:511] Master 127.7.132.190:39693 requested a full tablet report, sending...
I20260812 02:36:13.538266 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 9952
I20260812 02:36:13.538479 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502156971023-7698-0/minicluster-data/ts-0/wal/instance
I20260812 02:36:13.539155 7698 external_mini_cluster.cc:1428] Setting key c8691ed159e354f12518e9db17b0c5a2
I20260812 02:36:13.539731 9871 ts_manager.cc:194] Registered new tserver with Master: 2a89947c31174fd2a5b30285f3c23691 (127.7.132.129:45769)
I20260812 02:36:13.541460 9871 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.129@KRBTEST.COM'} at 127.7.132.129:43801
WARNING: no policy specified for kudu/127.7.132.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.130.keytab.
Entry for principal kudu/127.7.132.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.130.keytab.
I20260812 02:36:13.585698 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502156971023-7698-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502156971023-7698-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502156971023-7698-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502156971023-7698-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.130:0
--local_ip_for_outbound_sockets=127.7.132.130
--webserver_interface=127.7.132.130
--webserver_port=0
--tserver_master_addrs=127.7.132.190:39693
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.130.keytab
--principal=kudu/127.7.132.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:34815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:36:13.927692 10121 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:36:13.927999 10121 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:36:13.928107 10121 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:36:13.935492 10121 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:36:13.935608 10121 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:36:13.935701 10121 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260812 02:36:13.935755 10121 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 02:36:13.935829 10121 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.130
I20260812 02:36:13.946022 10121 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:34815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502156971023-7698-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502156971023-7698-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.130.keytab
--principal=kudu/127.7.132.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502156971023-7698-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.132.130
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.132.190:39693
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.130
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502156971023-7698-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:36:13.947557 10121 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:36:13.949060 10121 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:36:13.962783 10128 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:36:13.962834 10129 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:36:13.963478 10132 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:36:13.964102 10121 server_base.cc:1034] running on GCE node
I20260812 02:36:13.964800 10121 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:36:13.965857 10121 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:36:13.967110 10121 hybrid_clock.cc:648] HybridClock initialized: now 1786502173967032 us; error 76 us; skew 500 ppm
I20260812 02:36:13.967492 10121 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:36:13 dist-test-slave-mn4r krb5kdc[9811](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502173, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:36:13.975952 10121 init.cc:377] Logged in from keytab as kudu/127.7.132.130@KRBTEST.COM (short username kudu)
I20260812 02:36:13.977783 10121 webserver.cc:460] Webserver started at http://127.7.132.130:36909/ using document root <none> and password file <none>
I20260812 02:36:13.978631 10121 fs_manager.cc:359] Metadata directory not provided
I20260812 02:36:13.978791 10121 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:36:13.979151 10121 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:36:13.982409 10121 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502156971023-7698-0/minicluster-data/ts-1/data/instance:
uuid: "56abc4db7e004e008c44748e7ba48a78"
format_stamp: "Formatted at 2026-08-12 02:36:13 on dist-test-slave-mn4r"
server_key: "c64fbe3fa355f258972834464200b300"
server_key_iv: "75fcc822ca359fc88dd6d395a9217ec4"
server_key_version: "encryptionkey@0"
I20260812 02:36:13.983242 10121 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502156971023-7698-0/minicluster-data/ts-1/wal/instance:
uuid: "56abc4db7e004e008c44748e7ba48a78"
format_stamp: "Formatted at 2026-08-12 02:36:13 on dist-test-slave-mn4r"
server_key: "c64fbe3fa355f258972834464200b300"
server_key_iv: "75fcc822ca359fc88dd6d395a9217ec4"
server_key_version: "encryptionkey@0"
I20260812 02:36:13.988986 10121 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260812 02:36:13.992877 10143 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:36:13.994639 10121 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 02:36:13.994819 10121 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502156971023-7698-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502156971023-7698-0/minicluster-data/ts-1/wal
uuid: "56abc4db7e004e008c44748e7ba48a78"
format_stamp: "Formatted at 2026-08-12 02:36:13 on dist-test-slave-mn4r"
server_key: "c64fbe3fa355f258972834464200b300"
server_key_iv: "75fcc822ca359fc88dd6d395a9217ec4"
server_key_version: "encryptionkey@0"
I20260812 02:36:13.995018 10121 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502156971023-7698-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502156971023-7698-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502156971023-7698-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:36:14.022524 10121 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:36:14.027259 10121 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:36:14.027633 10121 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:36:14.030485 10121 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:36:14.030644 10121 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:36:14.030771 10121 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:36:14.030855 10121 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 02:36:14 dist-test-slave-mn4r krb5kdc[9811](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502173, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.130@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 02:36:14.058349 9982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:14.031219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:43813 (local address 127.7.132.190:39693)
0812 02:36:14.031470 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:14.031480 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:14.033103 (+ 1623us) server_negotiation.cc:407] Connection header received
0812 02:36:14.035434 (+ 2331us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:14.035441 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:14.035547 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:14.035663 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:14.038629 (+ 2966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:14.039693 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:14.042371 (+ 2678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:14.043048 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:14.043199 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:14.048698 (+ 5499us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:14.048710 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:14.048718 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:14.048780 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:14.053316 (+ 4536us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:14.054483 (+ 1167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:14.054493 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:14.054499 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:14.054674 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:14.055266 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:14.055272 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:14.055275 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:14.055746 (+ 471us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:14.055928 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:14.057720 (+ 1792us) server_negotiation.cc:299] Negotiation successful
0812 02:36:14.057948 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 02:36:14.059451 10155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:14.031902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:39693 (local address 127.7.132.130:43813)
0812 02:36:14.032909 (+ 1007us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:14.032960 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:14.035117 (+ 2157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:14.035919 (+ 802us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:14.035930 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:14.037306 (+ 1376us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:36:14.038403 (+ 1097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:14.038436 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:14.039910 (+ 1474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:14.039919 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:14.042205 (+ 2286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:14.042219 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:14.043335 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:14.043343 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:14.043676 (+ 333us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:14.044728 (+ 1052us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:36:14.044757 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:36:14.048343 (+ 3586us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:36:14.053541 (+ 5198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:14.053549 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:14.053564 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:14.054304 (+ 740us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:14.054802 (+ 498us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:14.054808 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:14.054819 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:14.055124 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:14.055896 (+ 772us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:36:14.055908 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:36:14.056533 (+ 625us) client_negotiation.cc:769] Sending connection context
0812 02:36:14.056966 (+ 433us) client_negotiation.cc:240] Negotiation successful
0812 02:36:14.057637 (+ 671us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":691,"thread_start_us":430,"threads_started":1}
I20260812 02:36:14.085153 10248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:14.066685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:39693 (local address 127.7.132.130:41183)
0812 02:36:14.067622 (+ 937us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:14.067648 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:14.067840 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:14.068316 (+ 476us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:14.068324 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:14.069170 (+ 846us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:36:14.069681 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:14.069698 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:14.070924 (+ 1226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:14.070932 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:14.073126 (+ 2194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:14.073149 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:14.074740 (+ 1591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:14.074749 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:14.075028 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:14.075931 (+ 903us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:36:14.075965 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:36:14.077290 (+ 1325us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:36:14.081708 (+ 4418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:14.081714 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:14.081719 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:14.082493 (+ 774us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:14.083056 (+ 563us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:14.083061 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:14.083072 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:14.083280 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:14.084176 (+ 896us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:36:14.084182 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:36:14.084360 (+ 178us) client_negotiation.cc:769] Sending connection context
0812 02:36:14.084496 (+ 136us) client_negotiation.cc:240] Negotiation successful
0812 02:36:14.084801 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":721,"thread_start_us":532,"threads_started":1}
I20260812 02:36:14.085562 9982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:14.066679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:41183 (local address 127.7.132.190:39693)
0812 02:36:14.066950 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:14.066961 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:14.067766 (+ 805us) server_negotiation.cc:407] Connection header received
0812 02:36:14.068025 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:14.068029 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:14.068096 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:14.068237 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:14.069871 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:14.070772 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:14.073349 (+ 2577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:14.074213 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:14.074339 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:14.077510 (+ 3171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:14.077517 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:14.077521 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:14.077565 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:14.081525 (+ 3960us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:14.082712 (+ 1187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:14.082721 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:14.082727 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:14.082914 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:14.083429 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:14.083437 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:14.083443 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:14.084007 (+ 564us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:14.084173 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:14.084679 (+ 506us) server_negotiation.cc:299] Negotiation successful
0812 02:36:14.085084 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260812 02:36:14.088981 10121 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.130:35987
I20260812 02:36:14.089031 10292 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.130:35987 every 8 connection(s)
I20260812 02:36:14.090337 10121 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502156971023-7698-0/minicluster-data/ts-1/data/info.pb
I20260812 02:36:14.092506 10293 heartbeater.cc:347] Connected to a master server at 127.7.132.190:39693
I20260812 02:36:14.092746 10293 heartbeater.cc:464] Registering TS with master...
I20260812 02:36:14.093528 10293 heartbeater.cc:511] Master 127.7.132.190:39693 requested a full tablet report, sending...
I20260812 02:36:14.095000 9871 ts_manager.cc:194] Registered new tserver with Master: 56abc4db7e004e008c44748e7ba48a78 (127.7.132.130:35987)
I20260812 02:36:14.095877 9871 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.130@KRBTEST.COM'} at 127.7.132.130:43813
I20260812 02:36:14.097862 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 10121
I20260812 02:36:14.098047 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502156971023-7698-0/minicluster-data/ts-1/wal/instance
I20260812 02:36:14.098587 7698 external_mini_cluster.cc:1428] Setting key ec659415897fd872bd021e6c682a992a
WARNING: no policy specified for kudu/127.7.132.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.131.keytab.
Entry for principal kudu/127.7.132.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.131.keytab.
I20260812 02:36:14.149785 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502156971023-7698-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502156971023-7698-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502156971023-7698-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502156971023-7698-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.131:0
--local_ip_for_outbound_sockets=127.7.132.131
--webserver_interface=127.7.132.131
--webserver_port=0
--tserver_master_addrs=127.7.132.190:39693
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.131.keytab
--principal=kudu/127.7.132.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:34815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:36:14.496153 10297 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:36:14.496469 10297 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:36:14.496645 10297 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:36:14.504602 10297 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:36:14.504736 10297 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:36:14.504920 10297 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260812 02:36:14.505013 10297 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 02:36:14.505143 10297 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.131
I20260812 02:36:14.515931 10297 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:34815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502156971023-7698-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502156971023-7698-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.131.keytab
--principal=kudu/127.7.132.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502156971023-7698-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.132.131
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.132.190:39693
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.131
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502156971023-7698-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:36:14.517349 10297 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:36:14.518888 10297 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:36:14.533031 10304 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:36:14.532948 10303 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:36:14.533310 10306 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:36:14.534531 10297 server_base.cc:1034] running on GCE node
I20260812 02:36:14.535173 10297 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:36:14.536170 10297 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:36:14.537406 10297 hybrid_clock.cc:648] HybridClock initialized: now 1786502174537347 us; error 54 us; skew 500 ppm
I20260812 02:36:14.537760 10297 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:36:14 dist-test-slave-mn4r krb5kdc[9811](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502174, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:36:14.541102 10297 init.cc:377] Logged in from keytab as kudu/127.7.132.131@KRBTEST.COM (short username kudu)
I20260812 02:36:14.542753 10297 webserver.cc:460] Webserver started at http://127.7.132.131:38601/ using document root <none> and password file <none>
I20260812 02:36:14.543491 10297 fs_manager.cc:359] Metadata directory not provided
I20260812 02:36:14.543645 10297 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:36:14.543995 10297 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:36:14.547163 10117 heartbeater.cc:503] Master 127.7.132.190:39693 was elected leader, sending a full tablet report...
I20260812 02:36:14.547330 10297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502156971023-7698-0/minicluster-data/ts-2/data/instance:
uuid: "d3afa02e0d8841908636c0c61115dbf4"
format_stamp: "Formatted at 2026-08-12 02:36:14 on dist-test-slave-mn4r"
server_key: "1cc6d78a3c7af175cc0537c2dbe0f8b7"
server_key_iv: "09b9268f2106445d3dbe2b2b1550f0f7"
server_key_version: "encryptionkey@0"
I20260812 02:36:14.548390 10297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502156971023-7698-0/minicluster-data/ts-2/wal/instance:
uuid: "d3afa02e0d8841908636c0c61115dbf4"
format_stamp: "Formatted at 2026-08-12 02:36:14 on dist-test-slave-mn4r"
server_key: "1cc6d78a3c7af175cc0537c2dbe0f8b7"
server_key_iv: "09b9268f2106445d3dbe2b2b1550f0f7"
server_key_version: "encryptionkey@0"
I20260812 02:36:14.559239 10297 fs_manager.cc:691] Time spent creating directory manager: real 0.010s user 0.007s sys 0.005s
I20260812 02:36:14.564049 10313 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:36:14.565820 10297 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 02:36:14.566066 10297 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502156971023-7698-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502156971023-7698-0/minicluster-data/ts-2/wal
uuid: "d3afa02e0d8841908636c0c61115dbf4"
format_stamp: "Formatted at 2026-08-12 02:36:14 on dist-test-slave-mn4r"
server_key: "1cc6d78a3c7af175cc0537c2dbe0f8b7"
server_key_iv: "09b9268f2106445d3dbe2b2b1550f0f7"
server_key_version: "encryptionkey@0"
I20260812 02:36:14.566375 10297 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502156971023-7698-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502156971023-7698-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502156971023-7698-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:36:14.601718 10297 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:36:14.606237 10297 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:36:14.606501 10297 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:36:14.609440 10297 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:36:14.609540 10297 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:36:14.609696 10297 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:36:14.609756 10297 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 02:36:14 dist-test-slave-mn4r krb5kdc[9811](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502174, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.131@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 02:36:14.640812 10326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:14.610449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:57169 (local address 127.7.132.190:39693)
0812 02:36:14.611329 (+ 880us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:14.611337 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:14.612571 (+ 1234us) server_negotiation.cc:407] Connection header received
0812 02:36:14.615953 (+ 3382us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:14.615963 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:14.616072 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:14.616172 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:14.619494 (+ 3322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:14.621020 (+ 1526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:14.623751 (+ 2731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:14.624646 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:14.624834 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:14.630885 (+ 6051us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:14.630896 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:14.630912 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:14.630984 (+ 72us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:14.635726 (+ 4742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:14.636908 (+ 1182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:14.636916 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:14.636921 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:14.637128 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:14.637825 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:14.637834 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:14.637841 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:14.638393 (+ 552us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:14.638635 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:14.639873 (+ 1238us) server_negotiation.cc:299] Negotiation successful
0812 02:36:14.640196 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":740,"thread_start_us":620,"threads_started":1}
I20260812 02:36:14.642431 10327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:14.611220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:39693 (local address 127.7.132.131:57169)
0812 02:36:14.612339 (+ 1119us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:14.612411 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:14.615503 (+ 3092us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:14.616882 (+ 1379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:14.616899 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:14.618031 (+ 1132us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:36:14.619225 (+ 1194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:14.619259 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:14.621251 (+ 1992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:14.621261 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:14.623561 (+ 2300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:14.623575 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:14.624947 (+ 1372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:14.624956 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:14.625286 (+ 330us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:14.626192 (+ 906us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:36:14.626238 (+ 46us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:36:14.630445 (+ 4207us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:36:14.635928 (+ 5483us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:14.635942 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:14.635980 (+ 38us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:14.636725 (+ 745us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:14.637299 (+ 574us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:14.637307 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:14.637313 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:14.637660 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:14.638602 (+ 942us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:36:14.638614 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:36:14.639246 (+ 632us) client_negotiation.cc:769] Sending connection context
0812 02:36:14.639668 (+ 422us) client_negotiation.cc:240] Negotiation successful
0812 02:36:14.640410 (+ 742us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":767,"thread_start_us":469,"threads_started":1}
I20260812 02:36:14.657670 10297 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.131:39029
I20260812 02:36:14.657725 10466 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.131:39029 every 8 connection(s)
I20260812 02:36:14.659163 10297 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502156971023-7698-0/minicluster-data/ts-2/data/info.pb
I20260812 02:36:14.659516 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 10297
I20260812 02:36:14.659747 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502156971023-7698-0/minicluster-data/ts-2/wal/instance
I20260812 02:36:14.660331 7698 external_mini_cluster.cc:1428] Setting key 36ecfda01650db5fe62f1de8f1cad29d
I20260812 02:36:14.664350 10467 heartbeater.cc:347] Connected to a master server at 127.7.132.190:39693
I20260812 02:36:14.664678 10467 heartbeater.cc:464] Registering TS with master...
I20260812 02:36:14.665899 10467 heartbeater.cc:511] Master 127.7.132.190:39693 requested a full tablet report, sending...
I20260812 02:36:14.667721 9870 ts_manager.cc:194] Registered new tserver with Master: d3afa02e0d8841908636c0c61115dbf4 (127.7.132.131:39029)
I20260812 02:36:14.668802 9870 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.131@KRBTEST.COM'} at 127.7.132.131:57169
I20260812 02:36:14.672190 10459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:14.650318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:39693 (local address 127.7.132.131:52461)
0812 02:36:14.654786 (+ 4468us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:14.654822 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:14.655049 (+ 227us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:14.655526 (+ 477us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:14.655530 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:14.656018 (+ 488us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:36:14.656618 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:14.656633 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:14.658215 (+ 1582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:14.658219 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:14.659668 (+ 1449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:14.659689 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:14.662701 (+ 3012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:14.662712 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:14.662957 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:14.663773 (+ 816us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:36:14.663801 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:36:14.665145 (+ 1344us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:36:14.669211 (+ 4066us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:14.669216 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:14.669220 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:14.669844 (+ 624us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:14.670297 (+ 453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:14.670301 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:14.670304 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:14.670520 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:14.671161 (+ 641us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:36:14.671167 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:36:14.671355 (+ 188us) client_negotiation.cc:769] Sending connection context
0812 02:36:14.671559 (+ 204us) client_negotiation.cc:240] Negotiation successful
0812 02:36:14.671838 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4277,"thread_start_us":4075,"threads_started":1}
I20260812 02:36:14.672209 10326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:14.650126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:52461 (local address 127.7.132.190:39693)
0812 02:36:14.650348 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:14.650354 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:14.654954 (+ 4600us) server_negotiation.cc:407] Connection header received
0812 02:36:14.655230 (+ 276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:14.655239 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:14.655351 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:14.655464 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:14.656825 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:14.658054 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:14.659856 (+ 1802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:14.660873 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:14.661056 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:14.665380 (+ 4324us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:14.665388 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:14.665394 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:14.665439 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:14.669067 (+ 3628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:14.669980 (+ 913us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:14.669986 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:14.669990 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:14.670171 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:14.670643 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:14.670648 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:14.670652 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:14.670982 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:14.671127 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:14.671690 (+ 563us) server_negotiation.cc:299] Negotiation successful
0812 02:36:14.671934 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260812 02:36:14.677016 7698 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 02:36:14 dist-test-slave-mn4r krb5kdc[9811](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502174, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 02:36:14 dist-test-slave-mn4r krb5kdc[9811](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502174, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 02:36:14.733249 10326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:14.718979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49418 (local address 127.7.132.190:39693)
0812 02:36:14.719224 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:14.719231 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:14.719431 (+ 200us) server_negotiation.cc:407] Connection header received
0812 02:36:14.719673 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:14.719677 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:14.719745 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:14.719856 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:14.721285 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:14.722054 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:14.723366 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:14.723802 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:14.723944 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:14.728087 (+ 4143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:14.728092 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:14.728095 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:14.728132 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:14.730639 (+ 2507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:14.731288 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:14.731294 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:14.731297 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:14.731424 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:14.731817 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:14.731822 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:14.731825 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:14.732101 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:14.732375 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:14.732662 (+ 287us) server_negotiation.cc:299] Negotiation successful
0812 02:36:14.732899 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260812 02:36:14.737478 9870 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49418:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 02:36:14.745173 9870 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 02:36:14.782070 10482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:14.758822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.130:35987 (local address 127.0.0.1:51138)
0812 02:36:14.772115 (+ 13293us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:14.772136 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:14.772254 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:14.773520 (+ 1266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:14.773525 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:14.773557 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:36:14.774067 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:14.774107 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:14.775997 (+ 1890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:14.776002 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:14.778769 (+ 2767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:14.778788 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:14.780936 (+ 2148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:14.780963 (+ 27us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:14.781225 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:14.781276 (+ 51us) client_negotiation.cc:769] Sending connection context
0812 02:36:14.781367 (+ 91us) client_negotiation.cc:240] Negotiation successful
0812 02:36:14.781534 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13150,"mutex_wait_us":60,"thread_start_us":11721,"threads_started":1}
I20260812 02:36:14.782864 10487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:14.759474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51138 (local address 127.7.132.130:35987)
0812 02:36:14.772497 (+ 13023us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:14.772510 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:14.772543 (+ 33us) server_negotiation.cc:407] Connection header received
0812 02:36:14.772657 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:14.772665 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:14.772974 (+ 309us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:14.774362 (+ 1388us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:36:14.774611 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:14.775833 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:14.779010 (+ 3177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:14.780681 (+ 1671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:14.782016 (+ 1335us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:14.782283 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:14.782391 (+ 108us) server_negotiation.cc:299] Negotiation successful
0812 02:36:14.782596 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12827,"thread_start_us":13681,"threads_started":1}
I20260812 02:36:14.783063 10486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:14.759200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47862 (local address 127.7.132.131:39029)
0812 02:36:14.770526 (+ 11326us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:14.770541 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:14.772380 (+ 1839us) server_negotiation.cc:407] Connection header received
0812 02:36:14.772522 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:14.772531 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:14.772809 (+ 278us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:14.773070 (+ 261us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:36:14.773857 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:14.776112 (+ 2255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:14.778271 (+ 2159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:14.781463 (+ 3192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:14.781614 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:14.781867 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:14.782601 (+ 734us) server_negotiation.cc:299] Negotiation successful
0812 02:36:14.782754 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11027,"thread_start_us":10723,"threads_started":1}
I20260812 02:36:14.783550 10484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:14.758533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.131:39029 (local address 127.0.0.1:47862)
0812 02:36:14.771073 (+ 12540us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:14.771119 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:14.771363 (+ 244us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:14.773088 (+ 1725us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:14.773096 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:14.773129 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:36:14.773595 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:14.773611 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:14.776278 (+ 2667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:14.776284 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:14.778057 (+ 1773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:14.778075 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:14.781709 (+ 3634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:14.781721 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:14.781911 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:14.781946 (+ 35us) client_negotiation.cc:769] Sending connection context
0812 02:36:14.782059 (+ 113us) client_negotiation.cc:240] Negotiation successful
0812 02:36:14.782247 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12254,"thread_start_us":13296,"threads_started":1}
I20260812 02:36:14.785066 10488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:14.772717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49730 (local address 127.7.132.129:45769)
0812 02:36:14.773800 (+ 1083us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:14.773811 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:14.774643 (+ 832us) server_negotiation.cc:407] Connection header received
0812 02:36:14.774922 (+ 279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:14.774930 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:14.775201 (+ 271us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:14.775366 (+ 165us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:36:14.776370 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:14.777931 (+ 1561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:14.780521 (+ 2590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:14.782119 (+ 1598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:14.782405 (+ 286us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:14.782602 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:14.784508 (+ 1906us) server_negotiation.cc:299] Negotiation successful
0812 02:36:14.784707 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":907,"thread_start_us":760,"threads_started":1}
I20260812 02:36:14.786350 10489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:14.773838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:45769 (local address 127.0.0.1:49730)
0812 02:36:14.774528 (+ 690us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:14.774546 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:14.774746 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:14.775614 (+ 868us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:14.775618 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:14.775649 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:36:14.776105 (+ 456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:14.776117 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:14.778148 (+ 2031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:14.778155 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:14.780260 (+ 2105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:14.780275 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:14.784081 (+ 3806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:14.784086 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:14.784260 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:14.784291 (+ 31us) client_negotiation.cc:769] Sending connection context
0812 02:36:14.784388 (+ 97us) client_negotiation.cc:240] Negotiation successful
0812 02:36:14.784554 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":565,"thread_start_us":408,"threads_started":1}
I20260812 02:36:14.787197 10211 tablet_service.cc:1467] Processing CreateTablet for tablet f9a1d9b4cbbc40229f7602a233ce9162 (DEFAULT_TABLE table=test-table [id=fb27492d19874ab697b7f2e380edd197]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:36:14.787541 10381 tablet_service.cc:1467] Processing CreateTablet for tablet f9a1d9b4cbbc40229f7602a233ce9162 (DEFAULT_TABLE table=test-table [id=fb27492d19874ab697b7f2e380edd197]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:36:14.788616 10211 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f9a1d9b4cbbc40229f7602a233ce9162. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:36:14.788699 10381 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f9a1d9b4cbbc40229f7602a233ce9162. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:36:14.794656 10036 tablet_service.cc:1467] Processing CreateTablet for tablet f9a1d9b4cbbc40229f7602a233ce9162 (DEFAULT_TABLE table=test-table [id=fb27492d19874ab697b7f2e380edd197]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:36:14.798357 10036 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f9a1d9b4cbbc40229f7602a233ce9162. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:36:14.801910 10490 tablet_bootstrap.cc:492] T f9a1d9b4cbbc40229f7602a233ce9162 P d3afa02e0d8841908636c0c61115dbf4: Bootstrap starting.
I20260812 02:36:14.804229 10492 tablet_bootstrap.cc:492] T f9a1d9b4cbbc40229f7602a233ce9162 P 56abc4db7e004e008c44748e7ba48a78: Bootstrap starting.
I20260812 02:36:14.806041 10490 tablet_bootstrap.cc:654] T f9a1d9b4cbbc40229f7602a233ce9162 P d3afa02e0d8841908636c0c61115dbf4: Neither blocks nor log segments found. Creating new log.
I20260812 02:36:14.807523 10490 log.cc:824] T f9a1d9b4cbbc40229f7602a233ce9162 P d3afa02e0d8841908636c0c61115dbf4: Log is configured to *not* fsync() on all Append() calls
I20260812 02:36:14.807986 10492 tablet_bootstrap.cc:654] T f9a1d9b4cbbc40229f7602a233ce9162 P 56abc4db7e004e008c44748e7ba48a78: Neither blocks nor log segments found. Creating new log.
I20260812 02:36:14.809584 10492 log.cc:824] T f9a1d9b4cbbc40229f7602a233ce9162 P 56abc4db7e004e008c44748e7ba48a78: Log is configured to *not* fsync() on all Append() calls
I20260812 02:36:14.812966 10490 tablet_bootstrap.cc:492] T f9a1d9b4cbbc40229f7602a233ce9162 P d3afa02e0d8841908636c0c61115dbf4: No bootstrap required, opened a new log
I20260812 02:36:14.812971 10492 tablet_bootstrap.cc:492] T f9a1d9b4cbbc40229f7602a233ce9162 P 56abc4db7e004e008c44748e7ba48a78: No bootstrap required, opened a new log
I20260812 02:36:14.813279 10492 ts_tablet_manager.cc:1397] T f9a1d9b4cbbc40229f7602a233ce9162 P 56abc4db7e004e008c44748e7ba48a78: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260812 02:36:14.813279 10490 ts_tablet_manager.cc:1397] T f9a1d9b4cbbc40229f7602a233ce9162 P d3afa02e0d8841908636c0c61115dbf4: Time spent bootstrapping tablet: real 0.012s user 0.009s sys 0.000s
I20260812 02:36:14.814944 10496 tablet_bootstrap.cc:492] T f9a1d9b4cbbc40229f7602a233ce9162 P 2a89947c31174fd2a5b30285f3c23691: Bootstrap starting.
I20260812 02:36:14.818624 10496 tablet_bootstrap.cc:654] T f9a1d9b4cbbc40229f7602a233ce9162 P 2a89947c31174fd2a5b30285f3c23691: Neither blocks nor log segments found. Creating new log.
I20260812 02:36:14.819727 10496 log.cc:824] T f9a1d9b4cbbc40229f7602a233ce9162 P 2a89947c31174fd2a5b30285f3c23691: Log is configured to *not* fsync() on all Append() calls
I20260812 02:36:14.822747 10496 tablet_bootstrap.cc:492] T f9a1d9b4cbbc40229f7602a233ce9162 P 2a89947c31174fd2a5b30285f3c23691: No bootstrap required, opened a new log
I20260812 02:36:14.823051 10496 ts_tablet_manager.cc:1397] T f9a1d9b4cbbc40229f7602a233ce9162 P 2a89947c31174fd2a5b30285f3c23691: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260812 02:36:14.824612 10490 raft_consensus.cc:348] T f9a1d9b4cbbc40229f7602a233ce9162 P d3afa02e0d8841908636c0c61115dbf4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3afa02e0d8841908636c0c61115dbf4" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 39029 } } peers { permanent_uuid: "56abc4db7e004e008c44748e7ba48a78" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 35987 } } peers { permanent_uuid: "2a89947c31174fd2a5b30285f3c23691" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45769 } }
I20260812 02:36:14.824708 10492 raft_consensus.cc:348] T f9a1d9b4cbbc40229f7602a233ce9162 P 56abc4db7e004e008c44748e7ba48a78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3afa02e0d8841908636c0c61115dbf4" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 39029 } } peers { permanent_uuid: "56abc4db7e004e008c44748e7ba48a78" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 35987 } } peers { permanent_uuid: "2a89947c31174fd2a5b30285f3c23691" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45769 } }
I20260812 02:36:14.825120 10490 raft_consensus.cc:374] T f9a1d9b4cbbc40229f7602a233ce9162 P d3afa02e0d8841908636c0c61115dbf4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:36:14.825125 10492 raft_consensus.cc:374] T f9a1d9b4cbbc40229f7602a233ce9162 P 56abc4db7e004e008c44748e7ba48a78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:36:14.825223 10492 raft_consensus.cc:729] T f9a1d9b4cbbc40229f7602a233ce9162 P 56abc4db7e004e008c44748e7ba48a78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56abc4db7e004e008c44748e7ba48a78, State: Initialized, Role: FOLLOWER
I20260812 02:36:14.825224 10490 raft_consensus.cc:729] T f9a1d9b4cbbc40229f7602a233ce9162 P d3afa02e0d8841908636c0c61115dbf4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3afa02e0d8841908636c0c61115dbf4, State: Initialized, Role: FOLLOWER
I20260812 02:36:14.825881 10492 consensus_queue.cc:260] T f9a1d9b4cbbc40229f7602a233ce9162 P 56abc4db7e004e008c44748e7ba48a78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3afa02e0d8841908636c0c61115dbf4" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 39029 } } peers { permanent_uuid: "56abc4db7e004e008c44748e7ba48a78" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 35987 } } peers { permanent_uuid: "2a89947c31174fd2a5b30285f3c23691" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45769 } }
I20260812 02:36:14.825881 10490 consensus_queue.cc:260] T f9a1d9b4cbbc40229f7602a233ce9162 P d3afa02e0d8841908636c0c61115dbf4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3afa02e0d8841908636c0c61115dbf4" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 39029 } } peers { permanent_uuid: "56abc4db7e004e008c44748e7ba48a78" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 35987 } } peers { permanent_uuid: "2a89947c31174fd2a5b30285f3c23691" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45769 } }
I20260812 02:36:14.827633 10467 heartbeater.cc:503] Master 127.7.132.190:39693 was elected leader, sending a full tablet report...
I20260812 02:36:14.827685 10293 heartbeater.cc:503] Master 127.7.132.190:39693 was elected leader, sending a full tablet report...
I20260812 02:36:14.828336 10490 ts_tablet_manager.cc:1428] T f9a1d9b4cbbc40229f7602a233ce9162 P d3afa02e0d8841908636c0c61115dbf4: Time spent starting tablet: real 0.015s user 0.013s sys 0.004s
I20260812 02:36:14.829694 10492 ts_tablet_manager.cc:1428] T f9a1d9b4cbbc40229f7602a233ce9162 P 56abc4db7e004e008c44748e7ba48a78: Time spent starting tablet: real 0.016s user 0.012s sys 0.004s
I20260812 02:36:14.833854 10496 raft_consensus.cc:348] T f9a1d9b4cbbc40229f7602a233ce9162 P 2a89947c31174fd2a5b30285f3c23691 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3afa02e0d8841908636c0c61115dbf4" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 39029 } } peers { permanent_uuid: "56abc4db7e004e008c44748e7ba48a78" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 35987 } } peers { permanent_uuid: "2a89947c31174fd2a5b30285f3c23691" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45769 } }
I20260812 02:36:14.834420 10496 raft_consensus.cc:374] T f9a1d9b4cbbc40229f7602a233ce9162 P 2a89947c31174fd2a5b30285f3c23691 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:36:14.834529 10496 raft_consensus.cc:729] T f9a1d9b4cbbc40229f7602a233ce9162 P 2a89947c31174fd2a5b30285f3c23691 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a89947c31174fd2a5b30285f3c23691, State: Initialized, Role: FOLLOWER
I20260812 02:36:14.835217 10496 consensus_queue.cc:260] T f9a1d9b4cbbc40229f7602a233ce9162 P 2a89947c31174fd2a5b30285f3c23691 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3afa02e0d8841908636c0c61115dbf4" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 39029 } } peers { permanent_uuid: "56abc4db7e004e008c44748e7ba48a78" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 35987 } } peers { permanent_uuid: "2a89947c31174fd2a5b30285f3c23691" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45769 } }
I20260812 02:36:14.836727 10496 ts_tablet_manager.cc:1428] T f9a1d9b4cbbc40229f7602a233ce9162 P 2a89947c31174fd2a5b30285f3c23691: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
W20260812 02:36:14.842460 10294 tablet.cc:2369] T f9a1d9b4cbbc40229f7602a233ce9162 P 56abc4db7e004e008c44748e7ba48a78: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 02:36:14.916301 10468 tablet.cc:2369] T f9a1d9b4cbbc40229f7602a233ce9162 P d3afa02e0d8841908636c0c61115dbf4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 02:36:15.037177 10118 tablet.cc:2369] T f9a1d9b4cbbc40229f7602a233ce9162 P 2a89947c31174fd2a5b30285f3c23691: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 02:36:15.109345 10501 raft_consensus.cc:482] T f9a1d9b4cbbc40229f7602a233ce9162 P 2a89947c31174fd2a5b30285f3c23691 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 02:36:15.109660 10501 raft_consensus.cc:504] T f9a1d9b4cbbc40229f7602a233ce9162 P 2a89947c31174fd2a5b30285f3c23691 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3afa02e0d8841908636c0c61115dbf4" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 39029 } } peers { permanent_uuid: "56abc4db7e004e008c44748e7ba48a78" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 35987 } } peers { permanent_uuid: "2a89947c31174fd2a5b30285f3c23691" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45769 } }
I20260812 02:36:15.111901 10501 leader_election.cc:288] T f9a1d9b4cbbc40229f7602a233ce9162 P 2a89947c31174fd2a5b30285f3c23691 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3afa02e0d8841908636c0c61115dbf4 (127.7.132.131:39029), 56abc4db7e004e008c44748e7ba48a78 (127.7.132.130:35987)
I20260812 02:36:15.122128 10486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:15.112390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:44985 (local address 127.7.132.131:39029)
0812 02:36:15.112618 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:15.112625 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:15.113755 (+ 1130us) server_negotiation.cc:407] Connection header received
0812 02:36:15.113912 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:15.113917 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:15.114000 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:15.114111 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:36:15.114945 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:15.116083 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:15.119545 (+ 3462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:15.120830 (+ 1285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:15.120994 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:15.121073 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:15.121658 (+ 585us) server_negotiation.cc:299] Negotiation successful
0812 02:36:15.121797 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 02:36:15.122170 10499 raft_consensus.cc:482] T f9a1d9b4cbbc40229f7602a233ce9162 P 56abc4db7e004e008c44748e7ba48a78 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 02:36:15.122568 10504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:15.112299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.130:35987 (local address 127.7.132.129:45507)
0812 02:36:15.113244 (+ 945us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:15.113267 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:15.113512 (+ 245us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:15.114125 (+ 613us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:15.114132 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:15.114165 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:36:15.114713 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:15.114732 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:15.116248 (+ 1516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:15.116255 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:15.119339 (+ 3084us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:15.119356 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:15.121102 (+ 1746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:15.121110 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:15.121407 (+ 297us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:15.121504 (+ 97us) client_negotiation.cc:769] Sending connection context
0812 02:36:15.121613 (+ 109us) client_negotiation.cc:240] Negotiation successful
0812 02:36:15.122143 (+ 530us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":743,"mutex_wait_us":611,"thread_start_us":932,"threads_started":1}
I20260812 02:36:15.122774 10503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:15.112379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.131:39029 (local address 127.7.132.129:44985)
0812 02:36:15.113611 (+ 1232us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:15.113642 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:15.113780 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:15.114138 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:15.114142 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:15.114195 (+ 53us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:36:15.114772 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:15.114784 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:15.116249 (+ 1465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:15.116255 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:15.119343 (+ 3088us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:15.119357 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:15.121121 (+ 1764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:15.121128 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:15.121409 (+ 281us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:15.121504 (+ 95us) client_negotiation.cc:769] Sending connection context
0812 02:36:15.121613 (+ 109us) client_negotiation.cc:240] Negotiation successful
0812 02:36:15.121829 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1089,"mutex_wait_us":394,"spinlock_wait_cycles":35072,"thread_start_us":436,"threads_started":1}
I20260812 02:36:15.122521 10499 raft_consensus.cc:504] T f9a1d9b4cbbc40229f7602a233ce9162 P 56abc4db7e004e008c44748e7ba48a78 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3afa02e0d8841908636c0c61115dbf4" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 39029 } } peers { permanent_uuid: "56abc4db7e004e008c44748e7ba48a78" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 35987 } } peers { permanent_uuid: "2a89947c31174fd2a5b30285f3c23691" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45769 } }
I20260812 02:36:15.122167 10487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:15.112769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:45507 (local address 127.7.132.130:35987)
0812 02:36:15.113065 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:15.113071 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:15.113414 (+ 343us) server_negotiation.cc:407] Connection header received
0812 02:36:15.113783 (+ 369us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:15.113792 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:15.113902 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:15.114016 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:36:15.114913 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:15.116081 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:15.119537 (+ 3456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:15.120805 (+ 1268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:15.120993 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:15.121116 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:15.121658 (+ 542us) server_negotiation.cc:299] Negotiation successful
0812 02:36:15.121797 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260812 02:36:15.124039 10408 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9a1d9b4cbbc40229f7602a233ce9162" candidate_uuid: "2a89947c31174fd2a5b30285f3c23691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3afa02e0d8841908636c0c61115dbf4" is_pre_election: true
I20260812 02:36:15.124130 10235 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9a1d9b4cbbc40229f7602a233ce9162" candidate_uuid: "2a89947c31174fd2a5b30285f3c23691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56abc4db7e004e008c44748e7ba48a78" is_pre_election: true
I20260812 02:36:15.124655 10408 raft_consensus.cc:2454] T f9a1d9b4cbbc40229f7602a233ce9162 P d3afa02e0d8841908636c0c61115dbf4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a89947c31174fd2a5b30285f3c23691 in term 0.
I20260812 02:36:15.124615 10235 raft_consensus.cc:2454] T f9a1d9b4cbbc40229f7602a233ce9162 P 56abc4db7e004e008c44748e7ba48a78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a89947c31174fd2a5b30285f3c23691 in term 0.
I20260812 02:36:15.124748 10499 leader_election.cc:288] T f9a1d9b4cbbc40229f7602a233ce9162 P 56abc4db7e004e008c44748e7ba48a78 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3afa02e0d8841908636c0c61115dbf4 (127.7.132.131:39029), 2a89947c31174fd2a5b30285f3c23691 (127.7.132.129:45769)
I20260812 02:36:15.125519 9975 leader_election.cc:302] T f9a1d9b4cbbc40229f7602a233ce9162 P 2a89947c31174fd2a5b30285f3c23691 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a89947c31174fd2a5b30285f3c23691, d3afa02e0d8841908636c0c61115dbf4; no voters:
I20260812 02:36:15.126223 10501 raft_consensus.cc:2781] T f9a1d9b4cbbc40229f7602a233ce9162 P 2a89947c31174fd2a5b30285f3c23691 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 02:36:15.126398 10501 raft_consensus.cc:482] T f9a1d9b4cbbc40229f7602a233ce9162 P 2a89947c31174fd2a5b30285f3c23691 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 02:36:15.126534 10501 raft_consensus.cc:3037] T f9a1d9b4cbbc40229f7602a233ce9162 P 2a89947c31174fd2a5b30285f3c23691 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:36:15.130854 10501 raft_consensus.cc:504] T f9a1d9b4cbbc40229f7602a233ce9162 P 2a89947c31174fd2a5b30285f3c23691 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3afa02e0d8841908636c0c61115dbf4" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 39029 } } peers { permanent_uuid: "56abc4db7e004e008c44748e7ba48a78" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 35987 } } peers { permanent_uuid: "2a89947c31174fd2a5b30285f3c23691" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45769 } }
I20260812 02:36:15.131937 10486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:15.125299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:48155 (local address 127.7.132.131:39029)
0812 02:36:15.125515 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:15.125521 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:15.125967 (+ 446us) server_negotiation.cc:407] Connection header received
0812 02:36:15.126114 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:15.126119 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:15.126198 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:15.126296 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:36:15.126956 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:15.127855 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:15.129711 (+ 1856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:15.130884 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:15.131050 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:15.131255 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:15.131476 (+ 221us) server_negotiation.cc:299] Negotiation successful
0812 02:36:15.131615 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260812 02:36:15.132018 10506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:15.125120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.131:39029 (local address 127.7.132.130:48155)
0812 02:36:15.125775 (+ 655us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:15.125796 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:15.125952 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:15.126317 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:15.126322 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:15.126352 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:36:15.126741 (+ 389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:15.126776 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:15.128060 (+ 1284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:15.128063 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:15.129542 (+ 1479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:15.129555 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:15.131076 (+ 1521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:15.131082 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:15.131269 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:15.131311 (+ 42us) client_negotiation.cc:769] Sending connection context
0812 02:36:15.131444 (+ 133us) client_negotiation.cc:240] Negotiation successful
0812 02:36:15.131635 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":537,"thread_start_us":640,"threads_started":1}
I20260812 02:36:15.132237 10408 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9a1d9b4cbbc40229f7602a233ce9162" candidate_uuid: "2a89947c31174fd2a5b30285f3c23691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3afa02e0d8841908636c0c61115dbf4"
I20260812 02:36:15.132488 10408 raft_consensus.cc:3037] T f9a1d9b4cbbc40229f7602a233ce9162 P d3afa02e0d8841908636c0c61115dbf4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:36:15.133780 10407 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9a1d9b4cbbc40229f7602a233ce9162" candidate_uuid: "56abc4db7e004e008c44748e7ba48a78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3afa02e0d8841908636c0c61115dbf4" is_pre_election: true
I20260812 02:36:15.135855 10501 leader_election.cc:288] T f9a1d9b4cbbc40229f7602a233ce9162 P 2a89947c31174fd2a5b30285f3c23691 [CANDIDATE]: Term 1 election: Requested vote from peers d3afa02e0d8841908636c0c61115dbf4 (127.7.132.131:39029), 56abc4db7e004e008c44748e7ba48a78 (127.7.132.130:35987)
I20260812 02:36:15.136174 10408 raft_consensus.cc:2454] T f9a1d9b4cbbc40229f7602a233ce9162 P d3afa02e0d8841908636c0c61115dbf4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a89947c31174fd2a5b30285f3c23691 in term 1.
I20260812 02:36:15.136526 10235 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9a1d9b4cbbc40229f7602a233ce9162" candidate_uuid: "2a89947c31174fd2a5b30285f3c23691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56abc4db7e004e008c44748e7ba48a78"
I20260812 02:36:15.136636 9975 leader_election.cc:302] T f9a1d9b4cbbc40229f7602a233ce9162 P 2a89947c31174fd2a5b30285f3c23691 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a89947c31174fd2a5b30285f3c23691, d3afa02e0d8841908636c0c61115dbf4; no voters:
I20260812 02:36:15.136785 10235 raft_consensus.cc:3037] T f9a1d9b4cbbc40229f7602a233ce9162 P 56abc4db7e004e008c44748e7ba48a78 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:36:15.137074 10501 raft_consensus.cc:2781] T f9a1d9b4cbbc40229f7602a233ce9162 P 2a89947c31174fd2a5b30285f3c23691 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:36:15.137988 10488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:15.126714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:43201 (local address 127.7.132.129:45769)
0812 02:36:15.126963 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:15.126969 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:15.130780 (+ 3811us) server_negotiation.cc:407] Connection header received
0812 02:36:15.130978 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:15.130982 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:15.131062 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:15.131159 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:36:15.131983 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:15.133014 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:15.135617 (+ 2603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:15.136884 (+ 1267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:15.137082 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:15.137173 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:15.137593 (+ 420us) server_negotiation.cc:299] Negotiation successful
0812 02:36:15.137736 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260812 02:36:15.138270 10507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:15.126273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:45769 (local address 127.7.132.130:43201)
0812 02:36:15.130643 (+ 4370us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:15.130664 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:15.130817 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:15.131332 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:15.131337 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:15.131367 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:36:15.131767 (+ 400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:15.131783 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:15.133367 (+ 1584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:15.133372 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:15.135426 (+ 2054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:15.135444 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:15.137137 (+ 1693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:15.137145 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:15.137349 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:15.137407 (+ 58us) client_negotiation.cc:769] Sending connection context
0812 02:36:15.137545 (+ 138us) client_negotiation.cc:240] Negotiation successful
0812 02:36:15.137789 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4226,"thread_start_us":4005,"threads_started":1}
I20260812 02:36:15.139453 10061 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9a1d9b4cbbc40229f7602a233ce9162" candidate_uuid: "56abc4db7e004e008c44748e7ba48a78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a89947c31174fd2a5b30285f3c23691" is_pre_election: true
I20260812 02:36:15.140169 10235 raft_consensus.cc:2454] T f9a1d9b4cbbc40229f7602a233ce9162 P 56abc4db7e004e008c44748e7ba48a78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a89947c31174fd2a5b30285f3c23691 in term 1.
I20260812 02:36:15.142369 10501 raft_consensus.cc:686] T f9a1d9b4cbbc40229f7602a233ce9162 P 2a89947c31174fd2a5b30285f3c23691 [term 1 LEADER]: Becoming Leader. State: Replica: 2a89947c31174fd2a5b30285f3c23691, State: Running, Role: LEADER
I20260812 02:36:15.142944 10501 consensus_queue.cc:237] T f9a1d9b4cbbc40229f7602a233ce9162 P 2a89947c31174fd2a5b30285f3c23691 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3afa02e0d8841908636c0c61115dbf4" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 39029 } } peers { permanent_uuid: "56abc4db7e004e008c44748e7ba48a78" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 35987 } } peers { permanent_uuid: "2a89947c31174fd2a5b30285f3c23691" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45769 } }
I20260812 02:36:15.147569 10147 leader_election.cc:302] T f9a1d9b4cbbc40229f7602a233ce9162 P 56abc4db7e004e008c44748e7ba48a78 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56abc4db7e004e008c44748e7ba48a78; no voters: 2a89947c31174fd2a5b30285f3c23691, d3afa02e0d8841908636c0c61115dbf4
I20260812 02:36:15.148109 10499 raft_consensus.cc:2726] T f9a1d9b4cbbc40229f7602a233ce9162 P 56abc4db7e004e008c44748e7ba48a78 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 02:36:15.148993 9870 catalog_manager.cc:5495] T f9a1d9b4cbbc40229f7602a233ce9162 P 2a89947c31174fd2a5b30285f3c23691 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2a89947c31174fd2a5b30285f3c23691 (127.7.132.129). New cstate: current_term: 1 leader_uuid: "2a89947c31174fd2a5b30285f3c23691" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3afa02e0d8841908636c0c61115dbf4" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 39029 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56abc4db7e004e008c44748e7ba48a78" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 35987 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2a89947c31174fd2a5b30285f3c23691" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45769 } health_report { overall_health: HEALTHY } } }
Aug 12 02:36:15 dist-test-slave-mn4r krb5kdc[9811](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502172, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.190@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 02:36:15.276986 10517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:15.260892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:39693 (local address 127.0.0.1:49428)
0812 02:36:15.262117 (+ 1225us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:15.262137 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:15.262314 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:15.262771 (+ 457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:15.262775 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:15.263540 (+ 765us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:36:15.263817 (+ 277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:15.263826 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:15.265218 (+ 1392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:15.265222 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:15.266505 (+ 1283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:15.266519 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:15.267288 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:15.267293 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:15.267426 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:15.268188 (+ 762us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:36:15.268212 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:36:15.270749 (+ 2537us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:36:15.274248 (+ 3499us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:15.274261 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:15.274268 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:15.274789 (+ 521us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:15.275248 (+ 459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:15.275251 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:15.275254 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:15.275406 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:15.276051 (+ 645us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:36:15.276057 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:36:15.276240 (+ 183us) client_negotiation.cc:769] Sending connection context
0812 02:36:15.276386 (+ 146us) client_negotiation.cc:240] Negotiation successful
0812 02:36:15.276636 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1016,"thread_start_us":763,"threads_started":1}
I20260812 02:36:15.277275 10516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:15.260821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49428 (local address 127.7.132.190:39693)
0812 02:36:15.261767 (+ 946us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:15.261863 (+ 96us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:15.262286 (+ 423us) server_negotiation.cc:407] Connection header received
0812 02:36:15.262478 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:15.262483 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:15.262571 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:15.262703 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:15.264040 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:15.265080 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:15.266649 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:15.267100 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:15.267201 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:15.271043 (+ 3842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:15.271049 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:15.271053 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:15.271091 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:15.273999 (+ 2908us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:15.274970 (+ 971us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:15.274974 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:15.274977 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:15.275129 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:15.275575 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:15.275579 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:15.275581 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:15.275876 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:15.276041 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:15.276544 (+ 503us) server_negotiation.cc:299] Negotiation successful
0812 02:36:15.276803 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":691,"thread_start_us":405,"threads_started":1}
I20260812 02:36:15.283458 9870 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:49428:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260812 02:36:15.284654 9870 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 02:36:15.291985 10036 tablet_service.cc:1467] Processing CreateTablet for tablet 703b555fa81343d784de4166742d4fea (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d6941bddead8441aae3fff711eb4bfb8]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 02:36:15.292023 10211 tablet_service.cc:1467] Processing CreateTablet for tablet 703b555fa81343d784de4166742d4fea (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d6941bddead8441aae3fff711eb4bfb8]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 02:36:15.292330 10381 tablet_service.cc:1467] Processing CreateTablet for tablet 703b555fa81343d784de4166742d4fea (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d6941bddead8441aae3fff711eb4bfb8]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 02:36:15.293056 10036 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 703b555fa81343d784de4166742d4fea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:36:15.293068 10211 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 703b555fa81343d784de4166742d4fea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:36:15.293108 10381 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 703b555fa81343d784de4166742d4fea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:36:15.299242 10496 tablet_bootstrap.cc:492] T 703b555fa81343d784de4166742d4fea P 2a89947c31174fd2a5b30285f3c23691: Bootstrap starting.
I20260812 02:36:15.300639 10490 tablet_bootstrap.cc:492] T 703b555fa81343d784de4166742d4fea P d3afa02e0d8841908636c0c61115dbf4: Bootstrap starting.
I20260812 02:36:15.301048 10496 tablet_bootstrap.cc:654] T 703b555fa81343d784de4166742d4fea P 2a89947c31174fd2a5b30285f3c23691: Neither blocks nor log segments found. Creating new log.
I20260812 02:36:15.302492 10490 tablet_bootstrap.cc:654] T 703b555fa81343d784de4166742d4fea P d3afa02e0d8841908636c0c61115dbf4: Neither blocks nor log segments found. Creating new log.
I20260812 02:36:15.303654 10496 tablet_bootstrap.cc:492] T 703b555fa81343d784de4166742d4fea P 2a89947c31174fd2a5b30285f3c23691: No bootstrap required, opened a new log
I20260812 02:36:15.303670 10492 tablet_bootstrap.cc:492] T 703b555fa81343d784de4166742d4fea P 56abc4db7e004e008c44748e7ba48a78: Bootstrap starting.
I20260812 02:36:15.303812 10496 ts_tablet_manager.cc:1397] T 703b555fa81343d784de4166742d4fea P 2a89947c31174fd2a5b30285f3c23691: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260812 02:36:15.304519 10496 raft_consensus.cc:348] T 703b555fa81343d784de4166742d4fea P 2a89947c31174fd2a5b30285f3c23691 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a89947c31174fd2a5b30285f3c23691" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45769 } } peers { permanent_uuid: "56abc4db7e004e008c44748e7ba48a78" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 35987 } } peers { permanent_uuid: "d3afa02e0d8841908636c0c61115dbf4" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 39029 } }
I20260812 02:36:15.304714 10496 raft_consensus.cc:374] T 703b555fa81343d784de4166742d4fea P 2a89947c31174fd2a5b30285f3c23691 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:36:15.304827 10496 raft_consensus.cc:729] T 703b555fa81343d784de4166742d4fea P 2a89947c31174fd2a5b30285f3c23691 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a89947c31174fd2a5b30285f3c23691, State: Initialized, Role: FOLLOWER
I20260812 02:36:15.305123 10492 tablet_bootstrap.cc:654] T 703b555fa81343d784de4166742d4fea P 56abc4db7e004e008c44748e7ba48a78: Neither blocks nor log segments found. Creating new log.
I20260812 02:36:15.305097 10496 consensus_queue.cc:260] T 703b555fa81343d784de4166742d4fea P 2a89947c31174fd2a5b30285f3c23691 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a89947c31174fd2a5b30285f3c23691" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45769 } } peers { permanent_uuid: "56abc4db7e004e008c44748e7ba48a78" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 35987 } } peers { permanent_uuid: "d3afa02e0d8841908636c0c61115dbf4" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 39029 } }
I20260812 02:36:15.305651 10496 ts_tablet_manager.cc:1428] T 703b555fa81343d784de4166742d4fea P 2a89947c31174fd2a5b30285f3c23691: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260812 02:36:15.305565 10501 tablet_replica.cc:439] T 703b555fa81343d784de4166742d4fea P 2a89947c31174fd2a5b30285f3c23691: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a89947c31174fd2a5b30285f3c23691" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45769 } } peers { permanent_uuid: "56abc4db7e004e008c44748e7ba48a78" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 35987 } } peers { permanent_uuid: "d3afa02e0d8841908636c0c61115dbf4" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 39029 } } }
I20260812 02:36:15.305872 10501 tablet_replica.cc:442] T 703b555fa81343d784de4166742d4fea P 2a89947c31174fd2a5b30285f3c23691: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 02:36:15.307826 10490 tablet_bootstrap.cc:492] T 703b555fa81343d784de4166742d4fea P d3afa02e0d8841908636c0c61115dbf4: No bootstrap required, opened a new log
I20260812 02:36:15.307991 10490 ts_tablet_manager.cc:1397] T 703b555fa81343d784de4166742d4fea P d3afa02e0d8841908636c0c61115dbf4: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.000s
I20260812 02:36:15.308408 10492 tablet_bootstrap.cc:492] T 703b555fa81343d784de4166742d4fea P 56abc4db7e004e008c44748e7ba48a78: No bootstrap required, opened a new log
I20260812 02:36:15.308655 10492 ts_tablet_manager.cc:1397] T 703b555fa81343d784de4166742d4fea P 56abc4db7e004e008c44748e7ba48a78: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260812 02:36:15.308835 10490 raft_consensus.cc:348] T 703b555fa81343d784de4166742d4fea P d3afa02e0d8841908636c0c61115dbf4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a89947c31174fd2a5b30285f3c23691" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45769 } } peers { permanent_uuid: "56abc4db7e004e008c44748e7ba48a78" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 35987 } } peers { permanent_uuid: "d3afa02e0d8841908636c0c61115dbf4" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 39029 } }
I20260812 02:36:15.309111 10490 raft_consensus.cc:374] T 703b555fa81343d784de4166742d4fea P d3afa02e0d8841908636c0c61115dbf4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:36:15.309221 10490 raft_consensus.cc:729] T 703b555fa81343d784de4166742d4fea P d3afa02e0d8841908636c0c61115dbf4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3afa02e0d8841908636c0c61115dbf4, State: Initialized, Role: FOLLOWER
I20260812 02:36:15.309508 10492 raft_consensus.cc:348] T 703b555fa81343d784de4166742d4fea P 56abc4db7e004e008c44748e7ba48a78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a89947c31174fd2a5b30285f3c23691" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45769 } } peers { permanent_uuid: "56abc4db7e004e008c44748e7ba48a78" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 35987 } } peers { permanent_uuid: "d3afa02e0d8841908636c0c61115dbf4" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 39029 } }
I20260812 02:36:15.309515 10490 consensus_queue.cc:260] T 703b555fa81343d784de4166742d4fea P d3afa02e0d8841908636c0c61115dbf4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a89947c31174fd2a5b30285f3c23691" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45769 } } peers { permanent_uuid: "56abc4db7e004e008c44748e7ba48a78" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 35987 } } peers { permanent_uuid: "d3afa02e0d8841908636c0c61115dbf4" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 39029 } }
I20260812 02:36:15.310261 10490 ts_tablet_manager.cc:1428] T 703b555fa81343d784de4166742d4fea P d3afa02e0d8841908636c0c61115dbf4: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260812 02:36:15.310127 10500 tablet_replica.cc:439] T 703b555fa81343d784de4166742d4fea P d3afa02e0d8841908636c0c61115dbf4: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a89947c31174fd2a5b30285f3c23691" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45769 } } peers { permanent_uuid: "56abc4db7e004e008c44748e7ba48a78" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 35987 } } peers { permanent_uuid: "d3afa02e0d8841908636c0c61115dbf4" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 39029 } } }
I20260812 02:36:15.310458 10500 tablet_replica.cc:442] T 703b555fa81343d784de4166742d4fea P d3afa02e0d8841908636c0c61115dbf4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 02:36:15.309788 10492 raft_consensus.cc:374] T 703b555fa81343d784de4166742d4fea P 56abc4db7e004e008c44748e7ba48a78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:36:15.310887 10492 raft_consensus.cc:729] T 703b555fa81343d784de4166742d4fea P 56abc4db7e004e008c44748e7ba48a78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56abc4db7e004e008c44748e7ba48a78, State: Initialized, Role: FOLLOWER
I20260812 02:36:15.311167 10492 consensus_queue.cc:260] T 703b555fa81343d784de4166742d4fea P 56abc4db7e004e008c44748e7ba48a78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a89947c31174fd2a5b30285f3c23691" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45769 } } peers { permanent_uuid: "56abc4db7e004e008c44748e7ba48a78" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 35987 } } peers { permanent_uuid: "d3afa02e0d8841908636c0c61115dbf4" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 39029 } }
I20260812 02:36:15.311705 10492 ts_tablet_manager.cc:1428] T 703b555fa81343d784de4166742d4fea P 56abc4db7e004e008c44748e7ba48a78: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260812 02:36:15.312114 10499 tablet_replica.cc:439] T 703b555fa81343d784de4166742d4fea P 56abc4db7e004e008c44748e7ba48a78: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a89947c31174fd2a5b30285f3c23691" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45769 } } peers { permanent_uuid: "56abc4db7e004e008c44748e7ba48a78" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 35987 } } peers { permanent_uuid: "d3afa02e0d8841908636c0c61115dbf4" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 39029 } } }
I20260812 02:36:15.312420 10499 tablet_replica.cc:442] T 703b555fa81343d784de4166742d4fea P 56abc4db7e004e008c44748e7ba48a78: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 02:36:15.339882 10500 raft_consensus.cc:482] T 703b555fa81343d784de4166742d4fea P d3afa02e0d8841908636c0c61115dbf4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 02:36:15.340183 10500 raft_consensus.cc:504] T 703b555fa81343d784de4166742d4fea P d3afa02e0d8841908636c0c61115dbf4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a89947c31174fd2a5b30285f3c23691" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45769 } } peers { permanent_uuid: "56abc4db7e004e008c44748e7ba48a78" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 35987 } } peers { permanent_uuid: "d3afa02e0d8841908636c0c61115dbf4" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 39029 } }
I20260812 02:36:15.341787 10500 leader_election.cc:288] T 703b555fa81343d784de4166742d4fea P d3afa02e0d8841908636c0c61115dbf4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2a89947c31174fd2a5b30285f3c23691 (127.7.132.129:45769), 56abc4db7e004e008c44748e7ba48a78 (127.7.132.130:35987)
I20260812 02:36:15.350853 10522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:15.341974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:45769 (local address 127.7.132.131:37261)
0812 02:36:15.342846 (+ 872us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:15.342867 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:15.343070 (+ 203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:15.343419 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:15.343424 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:15.343450 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:36:15.344255 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:15.344275 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:15.345726 (+ 1451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:15.345731 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:15.348125 (+ 2394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:15.348147 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:15.349881 (+ 1734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:15.349900 (+ 19us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:15.350119 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:15.350161 (+ 42us) client_negotiation.cc:769] Sending connection context
0812 02:36:15.350265 (+ 104us) client_negotiation.cc:240] Negotiation successful
0812 02:36:15.350414 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":724,"mutex_wait_us":4,"thread_start_us":845,"threads_started":1}
I20260812 02:36:15.350879 10488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:15.342171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:37261 (local address 127.7.132.129:45769)
0812 02:36:15.342391 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:15.342398 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:15.342984 (+ 586us) server_negotiation.cc:407] Connection header received
0812 02:36:15.343228 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:15.343233 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:15.343297 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:15.343406 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:36:15.344466 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:15.345564 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:15.348345 (+ 2781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:15.349682 (+ 1337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:15.349869 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:15.349977 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:15.350319 (+ 342us) server_negotiation.cc:299] Negotiation successful
0812 02:36:15.350495 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260812 02:36:15.351686 10061 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "703b555fa81343d784de4166742d4fea" candidate_uuid: "d3afa02e0d8841908636c0c61115dbf4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a89947c31174fd2a5b30285f3c23691" is_pre_election: true
I20260812 02:36:15.351922 10487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:15.342392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:51875 (local address 127.7.132.130:35987)
0812 02:36:15.342621 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:15.342628 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:15.343263 (+ 635us) server_negotiation.cc:407] Connection header received
0812 02:36:15.343465 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:15.343469 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:15.343531 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:15.343632 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:36:15.344224 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:15.345408 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:15.348626 (+ 3218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:15.349804 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:15.350816 (+ 1012us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:15.350901 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:15.351485 (+ 584us) server_negotiation.cc:299] Negotiation successful
0812 02:36:15.351638 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260812 02:36:15.351938 10521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:15.342161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.130:35987 (local address 127.7.132.131:51875)
0812 02:36:15.343149 (+ 988us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:15.343164 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:15.343322 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:15.343654 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:15.343659 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:15.343684 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:36:15.344036 (+ 352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:15.344069 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:15.345596 (+ 1527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:15.345605 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:15.348352 (+ 2747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:15.348393 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:15.351055 (+ 2662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:15.351062 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:15.351267 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:15.351317 (+ 50us) client_negotiation.cc:769] Sending connection context
0812 02:36:15.351445 (+ 128us) client_negotiation.cc:240] Negotiation successful
0812 02:36:15.351581 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":864,"mutex_wait_us":38,"thread_start_us":395,"threads_started":1}
I20260812 02:36:15.352027 10061 raft_consensus.cc:2454] T 703b555fa81343d784de4166742d4fea P 2a89947c31174fd2a5b30285f3c23691 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3afa02e0d8841908636c0c61115dbf4 in term 0.
I20260812 02:36:15.352595 10316 leader_election.cc:302] T 703b555fa81343d784de4166742d4fea P d3afa02e0d8841908636c0c61115dbf4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a89947c31174fd2a5b30285f3c23691, d3afa02e0d8841908636c0c61115dbf4; no voters:
I20260812 02:36:15.352741 10235 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "703b555fa81343d784de4166742d4fea" candidate_uuid: "d3afa02e0d8841908636c0c61115dbf4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56abc4db7e004e008c44748e7ba48a78" is_pre_election: true
I20260812 02:36:15.353030 10500 raft_consensus.cc:2781] T 703b555fa81343d784de4166742d4fea P d3afa02e0d8841908636c0c61115dbf4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 02:36:15.353042 10235 raft_consensus.cc:2454] T 703b555fa81343d784de4166742d4fea P 56abc4db7e004e008c44748e7ba48a78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3afa02e0d8841908636c0c61115dbf4 in term 0.
I20260812 02:36:15.353168 10500 raft_consensus.cc:482] T 703b555fa81343d784de4166742d4fea P d3afa02e0d8841908636c0c61115dbf4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 02:36:15.353281 10500 raft_consensus.cc:3037] T 703b555fa81343d784de4166742d4fea P d3afa02e0d8841908636c0c61115dbf4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:36:15.355640 10500 raft_consensus.cc:504] T 703b555fa81343d784de4166742d4fea P d3afa02e0d8841908636c0c61115dbf4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a89947c31174fd2a5b30285f3c23691" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45769 } } peers { permanent_uuid: "56abc4db7e004e008c44748e7ba48a78" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 35987 } } peers { permanent_uuid: "d3afa02e0d8841908636c0c61115dbf4" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 39029 } }
I20260812 02:36:15.356243 10500 leader_election.cc:288] T 703b555fa81343d784de4166742d4fea P d3afa02e0d8841908636c0c61115dbf4 [CANDIDATE]: Term 1 election: Requested vote from peers 2a89947c31174fd2a5b30285f3c23691 (127.7.132.129:45769), 56abc4db7e004e008c44748e7ba48a78 (127.7.132.130:35987)
I20260812 02:36:15.356825 10235 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "703b555fa81343d784de4166742d4fea" candidate_uuid: "d3afa02e0d8841908636c0c61115dbf4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56abc4db7e004e008c44748e7ba48a78"
I20260812 02:36:15.356825 10061 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "703b555fa81343d784de4166742d4fea" candidate_uuid: "d3afa02e0d8841908636c0c61115dbf4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a89947c31174fd2a5b30285f3c23691"
I20260812 02:36:15.357066 10235 raft_consensus.cc:3037] T 703b555fa81343d784de4166742d4fea P 56abc4db7e004e008c44748e7ba48a78 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:36:15.357141 10061 raft_consensus.cc:3037] T 703b555fa81343d784de4166742d4fea P 2a89947c31174fd2a5b30285f3c23691 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:36:15.360239 10061 raft_consensus.cc:2454] T 703b555fa81343d784de4166742d4fea P 2a89947c31174fd2a5b30285f3c23691 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3afa02e0d8841908636c0c61115dbf4 in term 1.
I20260812 02:36:15.360680 10235 raft_consensus.cc:2454] T 703b555fa81343d784de4166742d4fea P 56abc4db7e004e008c44748e7ba48a78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3afa02e0d8841908636c0c61115dbf4 in term 1.
I20260812 02:36:15.360690 10316 leader_election.cc:302] T 703b555fa81343d784de4166742d4fea P d3afa02e0d8841908636c0c61115dbf4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a89947c31174fd2a5b30285f3c23691, d3afa02e0d8841908636c0c61115dbf4; no voters:
I20260812 02:36:15.361060 10500 raft_consensus.cc:2781] T 703b555fa81343d784de4166742d4fea P d3afa02e0d8841908636c0c61115dbf4 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:36:15.361544 10500 raft_consensus.cc:686] T 703b555fa81343d784de4166742d4fea P d3afa02e0d8841908636c0c61115dbf4 [term 1 LEADER]: Becoming Leader. State: Replica: d3afa02e0d8841908636c0c61115dbf4, State: Running, Role: LEADER
I20260812 02:36:15.362262 10500 consensus_queue.cc:237] T 703b555fa81343d784de4166742d4fea P d3afa02e0d8841908636c0c61115dbf4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a89947c31174fd2a5b30285f3c23691" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45769 } } peers { permanent_uuid: "56abc4db7e004e008c44748e7ba48a78" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 35987 } } peers { permanent_uuid: "d3afa02e0d8841908636c0c61115dbf4" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 39029 } }
I20260812 02:36:15.366300 10523 tablet_replica.cc:439] T 703b555fa81343d784de4166742d4fea P d3afa02e0d8841908636c0c61115dbf4: TxnStatusTablet state changed. Reason: New leader d3afa02e0d8841908636c0c61115dbf4. Latest consensus state: current_term: 1 leader_uuid: "d3afa02e0d8841908636c0c61115dbf4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a89947c31174fd2a5b30285f3c23691" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45769 } } peers { permanent_uuid: "56abc4db7e004e008c44748e7ba48a78" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 35987 } } peers { permanent_uuid: "d3afa02e0d8841908636c0c61115dbf4" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 39029 } } }
I20260812 02:36:15.366737 10523 tablet_replica.cc:442] T 703b555fa81343d784de4166742d4fea P d3afa02e0d8841908636c0c61115dbf4: This TxnStatusTablet replica's current role is: LEADER
I20260812 02:36:15.367556 10527 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260812 02:36:15.367863 10527 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260812 02:36:15.367830 9870 catalog_manager.cc:5495] T 703b555fa81343d784de4166742d4fea P d3afa02e0d8841908636c0c61115dbf4 reported cstate change: term changed from 0 to 1, leader changed from <none> to d3afa02e0d8841908636c0c61115dbf4 (127.7.132.131). New cstate: current_term: 1 leader_uuid: "d3afa02e0d8841908636c0c61115dbf4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a89947c31174fd2a5b30285f3c23691" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45769 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56abc4db7e004e008c44748e7ba48a78" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 35987 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d3afa02e0d8841908636c0c61115dbf4" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 39029 } health_report { overall_health: HEALTHY } } }
I20260812 02:36:15.392262 10486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:15.385144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47864 (local address 127.7.132.131:39029)
0812 02:36:15.385346 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:15.385351 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:15.385430 (+ 79us) server_negotiation.cc:407] Connection header received
0812 02:36:15.385768 (+ 338us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:15.385772 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:15.385871 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:15.386051 (+ 180us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 02:36:15.386542 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:15.387344 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:15.389355 (+ 2011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:15.389945 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:15.390158 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:15.390689 (+ 531us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 02:36:15.391543 (+ 854us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 02:36:15.391679 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:15.391871 (+ 192us) server_negotiation.cc:299] Negotiation successful
0812 02:36:15.391974 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260812 02:36:15.392310 10517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:15.385057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.131:39029 (local address 127.0.0.1:47864)
0812 02:36:15.385327 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:15.385350 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:15.385514 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:15.386131 (+ 617us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:15.386136 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:15.386153 (+ 17us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 02:36:15.386368 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:15.386381 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:15.387490 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:15.387496 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:15.389178 (+ 1682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:15.389196 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:15.390161 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:15.390168 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:15.390386 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:15.390471 (+ 85us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 02:36:15.391674 (+ 1203us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 02:36:15.391701 (+ 27us) client_negotiation.cc:769] Sending connection context
0812 02:36:15.391807 (+ 106us) client_negotiation.cc:240] Negotiation successful
0812 02:36:15.391977 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64}
I20260812 02:36:15.401659 10235 raft_consensus.cc:1261] T 703b555fa81343d784de4166742d4fea P 56abc4db7e004e008c44748e7ba48a78 [term 1 FOLLOWER]: Refusing update from remote peer d3afa02e0d8841908636c0c61115dbf4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:36:15.402292 10061 raft_consensus.cc:1261] T 703b555fa81343d784de4166742d4fea P 2a89947c31174fd2a5b30285f3c23691 [term 1 FOLLOWER]: Refusing update from remote peer d3afa02e0d8841908636c0c61115dbf4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:36:15.403040 10523 consensus_queue.cc:1035] T 703b555fa81343d784de4166742d4fea P d3afa02e0d8841908636c0c61115dbf4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56abc4db7e004e008c44748e7ba48a78" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 35987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:36:15.403858 10523 consensus_queue.cc:1035] T 703b555fa81343d784de4166742d4fea P d3afa02e0d8841908636c0c61115dbf4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a89947c31174fd2a5b30285f3c23691" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45769 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:36:15.415282 10501 tablet_replica.cc:439] T 703b555fa81343d784de4166742d4fea P 2a89947c31174fd2a5b30285f3c23691: TxnStatusTablet state changed. Reason: New leader d3afa02e0d8841908636c0c61115dbf4. Latest consensus state: current_term: 1 leader_uuid: "d3afa02e0d8841908636c0c61115dbf4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a89947c31174fd2a5b30285f3c23691" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45769 } } peers { permanent_uuid: "56abc4db7e004e008c44748e7ba48a78" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 35987 } } peers { permanent_uuid: "d3afa02e0d8841908636c0c61115dbf4" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 39029 } } }
I20260812 02:36:15.415663 10501 tablet_replica.cc:442] T 703b555fa81343d784de4166742d4fea P 2a89947c31174fd2a5b30285f3c23691: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 02:36:15.422698 10523 tablet_replica.cc:439] T 703b555fa81343d784de4166742d4fea P d3afa02e0d8841908636c0c61115dbf4: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d3afa02e0d8841908636c0c61115dbf4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a89947c31174fd2a5b30285f3c23691" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45769 } } peers { permanent_uuid: "56abc4db7e004e008c44748e7ba48a78" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 35987 } } peers { permanent_uuid: "d3afa02e0d8841908636c0c61115dbf4" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 39029 } } }
I20260812 02:36:15.423195 10523 tablet_replica.cc:442] T 703b555fa81343d784de4166742d4fea P d3afa02e0d8841908636c0c61115dbf4: This TxnStatusTablet replica's current role is: LEADER
I20260812 02:36:15.426265 10533 tablet_replica.cc:439] T 703b555fa81343d784de4166742d4fea P d3afa02e0d8841908636c0c61115dbf4: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d3afa02e0d8841908636c0c61115dbf4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a89947c31174fd2a5b30285f3c23691" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45769 } } peers { permanent_uuid: "56abc4db7e004e008c44748e7ba48a78" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 35987 } } peers { permanent_uuid: "d3afa02e0d8841908636c0c61115dbf4" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 39029 } } }
I20260812 02:36:15.426719 10533 tablet_replica.cc:442] T 703b555fa81343d784de4166742d4fea P d3afa02e0d8841908636c0c61115dbf4: This TxnStatusTablet replica's current role is: LEADER
I20260812 02:36:15.430190 10499 tablet_replica.cc:439] T 703b555fa81343d784de4166742d4fea P 56abc4db7e004e008c44748e7ba48a78: TxnStatusTablet state changed. Reason: New leader d3afa02e0d8841908636c0c61115dbf4. Latest consensus state: current_term: 1 leader_uuid: "d3afa02e0d8841908636c0c61115dbf4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a89947c31174fd2a5b30285f3c23691" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45769 } } peers { permanent_uuid: "56abc4db7e004e008c44748e7ba48a78" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 35987 } } peers { permanent_uuid: "d3afa02e0d8841908636c0c61115dbf4" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 39029 } } }
I20260812 02:36:15.430552 10499 tablet_replica.cc:442] T 703b555fa81343d784de4166742d4fea P 56abc4db7e004e008c44748e7ba48a78: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 02:36:15.432487 10501 tablet_replica.cc:439] T 703b555fa81343d784de4166742d4fea P 2a89947c31174fd2a5b30285f3c23691: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d3afa02e0d8841908636c0c61115dbf4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a89947c31174fd2a5b30285f3c23691" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45769 } } peers { permanent_uuid: "56abc4db7e004e008c44748e7ba48a78" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 35987 } } peers { permanent_uuid: "d3afa02e0d8841908636c0c61115dbf4" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 39029 } } }
I20260812 02:36:15.432808 10501 tablet_replica.cc:442] T 703b555fa81343d784de4166742d4fea P 2a89947c31174fd2a5b30285f3c23691: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 02:36:15.433172 10535 mvcc.cc:205] Tried to move back new op lower bound from 7317512910433128448 to 7317512910288527360. Current Snapshot: MvccSnapshot[applied={T|T < 7317512910433128448}]
I20260812 02:36:15.437000 10499 tablet_replica.cc:439] T 703b555fa81343d784de4166742d4fea P 56abc4db7e004e008c44748e7ba48a78: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d3afa02e0d8841908636c0c61115dbf4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a89947c31174fd2a5b30285f3c23691" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45769 } } peers { permanent_uuid: "56abc4db7e004e008c44748e7ba48a78" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 35987 } } peers { permanent_uuid: "d3afa02e0d8841908636c0c61115dbf4" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 39029 } } }
I20260812 02:36:15.437314 10499 tablet_replica.cc:442] T 703b555fa81343d784de4166742d4fea P 56abc4db7e004e008c44748e7ba48a78: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 02:36:15.451263 10488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:15.443382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49740 (local address 127.7.132.129:45769)
0812 02:36:15.443619 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:15.443626 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:15.444147 (+ 521us) server_negotiation.cc:407] Connection header received
0812 02:36:15.444375 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:15.444381 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:15.444522 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:15.444703 (+ 181us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 02:36:15.445296 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:15.446273 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:15.448674 (+ 2401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:15.449256 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:15.449383 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:15.449916 (+ 533us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 02:36:15.450601 (+ 685us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 02:36:15.450726 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:15.450864 (+ 138us) server_negotiation.cc:299] Negotiation successful
0812 02:36:15.450948 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260812 02:36:15.470808 10486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:15.464425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:45715 (local address 127.7.132.131:39029)
0812 02:36:15.464620 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:15.464625 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:15.465242 (+ 617us) server_negotiation.cc:407] Connection header received
0812 02:36:15.465513 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:15.465518 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:15.465641 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:15.465745 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 02:36:15.466321 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:15.467186 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:15.468603 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:15.469230 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:15.469394 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:15.469924 (+ 530us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 02:36:15.470048 (+ 124us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 02:36:15.470187 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:15.470329 (+ 142us) server_negotiation.cc:299] Negotiation successful
0812 02:36:15.470415 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 02:36:15.470872 10549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:15.464405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.131:39029 (local address 127.7.132.129:45715)
0812 02:36:15.465131 (+ 726us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:15.465148 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:15.465340 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:15.465767 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:15.465772 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:15.465785 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 02:36:15.466155 (+ 370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:15.466167 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:15.467323 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:15.467330 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:15.468443 (+ 1113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:15.468455 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:15.469430 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:15.469437 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:15.469616 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:15.469679 (+ 63us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 02:36:15.470187 (+ 508us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 02:36:15.470220 (+ 33us) client_negotiation.cc:769] Sending connection context
0812 02:36:15.470306 (+ 86us) client_negotiation.cc:240] Negotiation successful
0812 02:36:15.470458 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":563,"thread_start_us":404,"threads_started":1}
I20260812 02:36:15.482777 10234 raft_consensus.cc:1261] T f9a1d9b4cbbc40229f7602a233ce9162 P 56abc4db7e004e008c44748e7ba48a78 [term 1 FOLLOWER]: Refusing update from remote peer 2a89947c31174fd2a5b30285f3c23691: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:36:15.482858 10408 raft_consensus.cc:1261] T f9a1d9b4cbbc40229f7602a233ce9162 P d3afa02e0d8841908636c0c61115dbf4 [term 1 FOLLOWER]: Refusing update from remote peer 2a89947c31174fd2a5b30285f3c23691: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:36:15.483525 10508 consensus_queue.cc:1035] T f9a1d9b4cbbc40229f7602a233ce9162 P 2a89947c31174fd2a5b30285f3c23691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56abc4db7e004e008c44748e7ba48a78" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 35987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:36:15.484192 10501 consensus_queue.cc:1035] T f9a1d9b4cbbc40229f7602a233ce9162 P 2a89947c31174fd2a5b30285f3c23691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3afa02e0d8841908636c0c61115dbf4" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 39029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:36:15.498839 10532 mvcc.cc:205] Tried to move back new op lower bound from 7317512910769688576 to 7317512909390225408. Current Snapshot: MvccSnapshot[applied={T|T < 7317512910769688576}]
I20260812 02:36:15.529772 10488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:15.522696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:38539 (local address 127.7.132.129:45769)
0812 02:36:15.522969 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:15.522977 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:15.523212 (+ 235us) server_negotiation.cc:407] Connection header received
0812 02:36:15.523486 (+ 274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:15.523492 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:15.523578 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:15.523686 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 02:36:15.524542 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:15.525320 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:15.527496 (+ 2176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:15.528194 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:15.528341 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:15.528916 (+ 575us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 02:36:15.529077 (+ 161us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 02:36:15.529176 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:15.529306 (+ 130us) server_negotiation.cc:299] Negotiation successful
0812 02:36:15.529395 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130}
I20260812 02:36:15.529908 10555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:15.522217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:45769 (local address 127.7.132.131:38539)
0812 02:36:15.523096 (+ 879us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:15.523134 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:15.523320 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:15.523709 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:15.523714 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:15.523728 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 02:36:15.524271 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:15.524288 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:15.525510 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:15.525514 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:15.527306 (+ 1792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:15.527323 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:15.528415 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:15.528424 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:15.528658 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:15.528721 (+ 63us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 02:36:15.529183 (+ 462us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 02:36:15.529198 (+ 15us) client_negotiation.cc:769] Sending connection context
0812 02:36:15.529285 (+ 87us) client_negotiation.cc:240] Negotiation successful
0812 02:36:15.529447 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":612,"thread_start_us":524,"threads_started":1}
I20260812 02:36:15.592615 9870 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49418:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 02:36:15.592996 9870 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49418:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 02:36:15.597898 9870 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1a4320c66f614d04a17f19477a85c319: Sending DeleteTablet for 3 replicas of tablet f9a1d9b4cbbc40229f7602a233ce9162
I20260812 02:36:15.599640 10036 tablet_service.cc:1514] Processing DeleteTablet for tablet f9a1d9b4cbbc40229f7602a233ce9162 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:36:15 UTC) from {username='kudu', principal='kudu/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:49730
I20260812 02:36:15.599733 10211 tablet_service.cc:1514] Processing DeleteTablet for tablet f9a1d9b4cbbc40229f7602a233ce9162 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:36:15 UTC) from {username='kudu', principal='kudu/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:51138
I20260812 02:36:15.600622 10381 tablet_service.cc:1514] Processing DeleteTablet for tablet f9a1d9b4cbbc40229f7602a233ce9162 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:36:15 UTC) from {username='kudu', principal='kudu/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:47862
I20260812 02:36:15.602578 9865 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49418:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 02:36:15.603619 9865 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 02:36:15.615617 10035 tablet_service.cc:1467] Processing CreateTablet for tablet be900cfd63104ae89c5e508581b77fb1 (DEFAULT_TABLE table=test-table [id=949a79fa70794b3991719d00e9e321e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:36:15.615777 10380 tablet_service.cc:1467] Processing CreateTablet for tablet be900cfd63104ae89c5e508581b77fb1 (DEFAULT_TABLE table=test-table [id=949a79fa70794b3991719d00e9e321e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:36:15.616303 10035 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be900cfd63104ae89c5e508581b77fb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:36:15.616329 10380 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be900cfd63104ae89c5e508581b77fb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:36:15.618692 10210 tablet_service.cc:1467] Processing CreateTablet for tablet be900cfd63104ae89c5e508581b77fb1 (DEFAULT_TABLE table=test-table [id=949a79fa70794b3991719d00e9e321e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:36:15.619339 10210 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be900cfd63104ae89c5e508581b77fb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:36:15.623766 10496 tablet_bootstrap.cc:492] T be900cfd63104ae89c5e508581b77fb1 P 2a89947c31174fd2a5b30285f3c23691: Bootstrap starting.
I20260812 02:36:15.625669 10496 tablet_bootstrap.cc:654] T be900cfd63104ae89c5e508581b77fb1 P 2a89947c31174fd2a5b30285f3c23691: Neither blocks nor log segments found. Creating new log.
I20260812 02:36:15.626542 10561 tablet_replica.cc:330] T f9a1d9b4cbbc40229f7602a233ce9162 P d3afa02e0d8841908636c0c61115dbf4: stopping tablet replica
I20260812 02:36:15.627259 10560 tablet_replica.cc:330] T f9a1d9b4cbbc40229f7602a233ce9162 P 56abc4db7e004e008c44748e7ba48a78: stopping tablet replica
I20260812 02:36:15.627923 10561 raft_consensus.cc:2229] T f9a1d9b4cbbc40229f7602a233ce9162 P d3afa02e0d8841908636c0c61115dbf4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 02:36:15.628538 10560 raft_consensus.cc:2229] T f9a1d9b4cbbc40229f7602a233ce9162 P 56abc4db7e004e008c44748e7ba48a78 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 02:36:15.629038 10561 raft_consensus.cc:2258] T f9a1d9b4cbbc40229f7602a233ce9162 P d3afa02e0d8841908636c0c61115dbf4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:36:15.629488 10560 raft_consensus.cc:2258] T f9a1d9b4cbbc40229f7602a233ce9162 P 56abc4db7e004e008c44748e7ba48a78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:36:15.634605 10560 ts_tablet_manager.cc:1905] T f9a1d9b4cbbc40229f7602a233ce9162 P 56abc4db7e004e008c44748e7ba48a78: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:36:15.635456 10559 tablet_replica.cc:330] T f9a1d9b4cbbc40229f7602a233ce9162 P 2a89947c31174fd2a5b30285f3c23691: stopping tablet replica
I20260812 02:36:15.636026 10559 raft_consensus.cc:2229] T f9a1d9b4cbbc40229f7602a233ce9162 P 2a89947c31174fd2a5b30285f3c23691 [term 1 LEADER]: Raft consensus shutting down.
I20260812 02:36:15.636253 10561 ts_tablet_manager.cc:1905] T f9a1d9b4cbbc40229f7602a233ce9162 P d3afa02e0d8841908636c0c61115dbf4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:36:15.641381 10490 tablet_bootstrap.cc:492] T be900cfd63104ae89c5e508581b77fb1 P d3afa02e0d8841908636c0c61115dbf4: Bootstrap starting.
I20260812 02:36:15.638028 10492 tablet_bootstrap.cc:492] T be900cfd63104ae89c5e508581b77fb1 P 56abc4db7e004e008c44748e7ba48a78: Bootstrap starting.
I20260812 02:36:15.643635 10492 tablet_bootstrap.cc:654] T be900cfd63104ae89c5e508581b77fb1 P 56abc4db7e004e008c44748e7ba48a78: Neither blocks nor log segments found. Creating new log.
I20260812 02:36:15.642971 10496 tablet_bootstrap.cc:492] T be900cfd63104ae89c5e508581b77fb1 P 2a89947c31174fd2a5b30285f3c23691: No bootstrap required, opened a new log
I20260812 02:36:15.643853 10490 tablet_bootstrap.cc:654] T be900cfd63104ae89c5e508581b77fb1 P d3afa02e0d8841908636c0c61115dbf4: Neither blocks nor log segments found. Creating new log.
I20260812 02:36:15.643936 10496 ts_tablet_manager.cc:1397] T be900cfd63104ae89c5e508581b77fb1 P 2a89947c31174fd2a5b30285f3c23691: Time spent bootstrapping tablet: real 0.020s user 0.000s sys 0.012s
I20260812 02:36:15.644836 10496 raft_consensus.cc:348] T be900cfd63104ae89c5e508581b77fb1 P 2a89947c31174fd2a5b30285f3c23691 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a89947c31174fd2a5b30285f3c23691" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45769 } } peers { permanent_uuid: "56abc4db7e004e008c44748e7ba48a78" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 35987 } } peers { permanent_uuid: "d3afa02e0d8841908636c0c61115dbf4" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 39029 } }
I20260812 02:36:15.645124 10496 raft_consensus.cc:374] T be900cfd63104ae89c5e508581b77fb1 P 2a89947c31174fd2a5b30285f3c23691 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:36:15.645218 10496 raft_consensus.cc:729] T be900cfd63104ae89c5e508581b77fb1 P 2a89947c31174fd2a5b30285f3c23691 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a89947c31174fd2a5b30285f3c23691, State: Initialized, Role: FOLLOWER
I20260812 02:36:15.645533 10496 consensus_queue.cc:260] T be900cfd63104ae89c5e508581b77fb1 P 2a89947c31174fd2a5b30285f3c23691 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a89947c31174fd2a5b30285f3c23691" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45769 } } peers { permanent_uuid: "56abc4db7e004e008c44748e7ba48a78" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 35987 } } peers { permanent_uuid: "d3afa02e0d8841908636c0c61115dbf4" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 39029 } }
I20260812 02:36:15.645952 10559 raft_consensus.cc:2258] T f9a1d9b4cbbc40229f7602a233ce9162 P 2a89947c31174fd2a5b30285f3c23691 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:36:15.653947 10560 ts_tablet_manager.cc:1918] T f9a1d9b4cbbc40229f7602a233ce9162 P 56abc4db7e004e008c44748e7ba48a78: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 02:36:15.653960 10559 ts_tablet_manager.cc:1905] T f9a1d9b4cbbc40229f7602a233ce9162 P 2a89947c31174fd2a5b30285f3c23691: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:36:15.654460 10496 ts_tablet_manager.cc:1428] T be900cfd63104ae89c5e508581b77fb1 P 2a89947c31174fd2a5b30285f3c23691: Time spent starting tablet: real 0.010s user 0.000s sys 0.002s
I20260812 02:36:15.657996 10561 ts_tablet_manager.cc:1918] T f9a1d9b4cbbc40229f7602a233ce9162 P d3afa02e0d8841908636c0c61115dbf4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 02:36:15.658238 10561 log.cc:1196] T f9a1d9b4cbbc40229f7602a233ce9162 P d3afa02e0d8841908636c0c61115dbf4: Deleting WAL directory at /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502156971023-7698-0/minicluster-data/ts-2/wal/wals/f9a1d9b4cbbc40229f7602a233ce9162
I20260812 02:36:15.658869 10561 ts_tablet_manager.cc:1939] T f9a1d9b4cbbc40229f7602a233ce9162 P d3afa02e0d8841908636c0c61115dbf4: Deleting consensus metadata
I20260812 02:36:15.655910 10492 tablet_bootstrap.cc:492] T be900cfd63104ae89c5e508581b77fb1 P 56abc4db7e004e008c44748e7ba48a78: No bootstrap required, opened a new log
I20260812 02:36:15.659798 10492 ts_tablet_manager.cc:1397] T be900cfd63104ae89c5e508581b77fb1 P 56abc4db7e004e008c44748e7ba48a78: Time spent bootstrapping tablet: real 0.022s user 0.010s sys 0.004s
I20260812 02:36:15.660771 10492 raft_consensus.cc:348] T be900cfd63104ae89c5e508581b77fb1 P 56abc4db7e004e008c44748e7ba48a78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a89947c31174fd2a5b30285f3c23691" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45769 } } peers { permanent_uuid: "56abc4db7e004e008c44748e7ba48a78" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 35987 } } peers { permanent_uuid: "d3afa02e0d8841908636c0c61115dbf4" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 39029 } }
I20260812 02:36:15.661099 10492 raft_consensus.cc:374] T be900cfd63104ae89c5e508581b77fb1 P 56abc4db7e004e008c44748e7ba48a78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:36:15.661192 10492 raft_consensus.cc:729] T be900cfd63104ae89c5e508581b77fb1 P 56abc4db7e004e008c44748e7ba48a78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56abc4db7e004e008c44748e7ba48a78, State: Initialized, Role: FOLLOWER
I20260812 02:36:15.661443 10492 consensus_queue.cc:260] T be900cfd63104ae89c5e508581b77fb1 P 56abc4db7e004e008c44748e7ba48a78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a89947c31174fd2a5b30285f3c23691" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45769 } } peers { permanent_uuid: "56abc4db7e004e008c44748e7ba48a78" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 35987 } } peers { permanent_uuid: "d3afa02e0d8841908636c0c61115dbf4" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 39029 } }
I20260812 02:36:15.662194 10492 ts_tablet_manager.cc:1428] T be900cfd63104ae89c5e508581b77fb1 P 56abc4db7e004e008c44748e7ba48a78: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 02:36:15.662405 10560 log.cc:1196] T f9a1d9b4cbbc40229f7602a233ce9162 P 56abc4db7e004e008c44748e7ba48a78: Deleting WAL directory at /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502156971023-7698-0/minicluster-data/ts-1/wal/wals/f9a1d9b4cbbc40229f7602a233ce9162
I20260812 02:36:15.663125 10560 ts_tablet_manager.cc:1939] T f9a1d9b4cbbc40229f7602a233ce9162 P 56abc4db7e004e008c44748e7ba48a78: Deleting consensus metadata
I20260812 02:36:15.664909 9850 catalog_manager.cc:4841] TS d3afa02e0d8841908636c0c61115dbf4 (127.7.132.131:39029): tablet f9a1d9b4cbbc40229f7602a233ce9162 (table test-table [id=fb27492d19874ab697b7f2e380edd197]) successfully deleted
I20260812 02:36:15.672855 9849 catalog_manager.cc:4841] TS 56abc4db7e004e008c44748e7ba48a78 (127.7.132.130:35987): tablet f9a1d9b4cbbc40229f7602a233ce9162 (table test-table [id=fb27492d19874ab697b7f2e380edd197]) successfully deleted
I20260812 02:36:15.676625 10559 ts_tablet_manager.cc:1918] T f9a1d9b4cbbc40229f7602a233ce9162 P 2a89947c31174fd2a5b30285f3c23691: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 02:36:15.676851 10559 log.cc:1196] T f9a1d9b4cbbc40229f7602a233ce9162 P 2a89947c31174fd2a5b30285f3c23691: Deleting WAL directory at /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502156971023-7698-0/minicluster-data/ts-0/wal/wals/f9a1d9b4cbbc40229f7602a233ce9162
I20260812 02:36:15.677446 10559 ts_tablet_manager.cc:1939] T f9a1d9b4cbbc40229f7602a233ce9162 P 2a89947c31174fd2a5b30285f3c23691: Deleting consensus metadata
I20260812 02:36:15.680022 10490 tablet_bootstrap.cc:492] T be900cfd63104ae89c5e508581b77fb1 P d3afa02e0d8841908636c0c61115dbf4: No bootstrap required, opened a new log
I20260812 02:36:15.680249 10490 ts_tablet_manager.cc:1397] T be900cfd63104ae89c5e508581b77fb1 P d3afa02e0d8841908636c0c61115dbf4: Time spent bootstrapping tablet: real 0.039s user 0.012s sys 0.000s
I20260812 02:36:15.681183 10490 raft_consensus.cc:348] T be900cfd63104ae89c5e508581b77fb1 P d3afa02e0d8841908636c0c61115dbf4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a89947c31174fd2a5b30285f3c23691" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45769 } } peers { permanent_uuid: "56abc4db7e004e008c44748e7ba48a78" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 35987 } } peers { permanent_uuid: "d3afa02e0d8841908636c0c61115dbf4" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 39029 } }
I20260812 02:36:15.681470 10490 raft_consensus.cc:374] T be900cfd63104ae89c5e508581b77fb1 P d3afa02e0d8841908636c0c61115dbf4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:36:15.681572 10490 raft_consensus.cc:729] T be900cfd63104ae89c5e508581b77fb1 P d3afa02e0d8841908636c0c61115dbf4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3afa02e0d8841908636c0c61115dbf4, State: Initialized, Role: FOLLOWER
I20260812 02:36:15.681924 10490 consensus_queue.cc:260] T be900cfd63104ae89c5e508581b77fb1 P d3afa02e0d8841908636c0c61115dbf4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a89947c31174fd2a5b30285f3c23691" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45769 } } peers { permanent_uuid: "56abc4db7e004e008c44748e7ba48a78" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 35987 } } peers { permanent_uuid: "d3afa02e0d8841908636c0c61115dbf4" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 39029 } }
I20260812 02:36:15.685257 10490 ts_tablet_manager.cc:1428] T be900cfd63104ae89c5e508581b77fb1 P d3afa02e0d8841908636c0c61115dbf4: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260812 02:36:15.689170 9850 catalog_manager.cc:4841] TS 2a89947c31174fd2a5b30285f3c23691 (127.7.132.129:45769): tablet f9a1d9b4cbbc40229f7602a233ce9162 (table test-table [id=fb27492d19874ab697b7f2e380edd197]) successfully deleted
I20260812 02:36:15.855610 10499 raft_consensus.cc:482] T be900cfd63104ae89c5e508581b77fb1 P 56abc4db7e004e008c44748e7ba48a78 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 02:36:15.855845 10499 raft_consensus.cc:504] T be900cfd63104ae89c5e508581b77fb1 P 56abc4db7e004e008c44748e7ba48a78 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a89947c31174fd2a5b30285f3c23691" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45769 } } peers { permanent_uuid: "56abc4db7e004e008c44748e7ba48a78" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 35987 } } peers { permanent_uuid: "d3afa02e0d8841908636c0c61115dbf4" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 39029 } }
I20260812 02:36:15.856633 10499 leader_election.cc:288] T be900cfd63104ae89c5e508581b77fb1 P 56abc4db7e004e008c44748e7ba48a78 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2a89947c31174fd2a5b30285f3c23691 (127.7.132.129:45769), d3afa02e0d8841908636c0c61115dbf4 (127.7.132.131:39029)
I20260812 02:36:15.857316 10061 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be900cfd63104ae89c5e508581b77fb1" candidate_uuid: "56abc4db7e004e008c44748e7ba48a78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a89947c31174fd2a5b30285f3c23691" is_pre_election: true
I20260812 02:36:15.857366 10407 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be900cfd63104ae89c5e508581b77fb1" candidate_uuid: "56abc4db7e004e008c44748e7ba48a78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3afa02e0d8841908636c0c61115dbf4" is_pre_election: true
I20260812 02:36:15.857636 10061 raft_consensus.cc:2454] T be900cfd63104ae89c5e508581b77fb1 P 2a89947c31174fd2a5b30285f3c23691 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56abc4db7e004e008c44748e7ba48a78 in term 0.
I20260812 02:36:15.857643 10407 raft_consensus.cc:2454] T be900cfd63104ae89c5e508581b77fb1 P d3afa02e0d8841908636c0c61115dbf4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56abc4db7e004e008c44748e7ba48a78 in term 0.
I20260812 02:36:15.858217 10147 leader_election.cc:302] T be900cfd63104ae89c5e508581b77fb1 P 56abc4db7e004e008c44748e7ba48a78 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a89947c31174fd2a5b30285f3c23691, 56abc4db7e004e008c44748e7ba48a78; no voters:
I20260812 02:36:15.858527 10499 raft_consensus.cc:2781] T be900cfd63104ae89c5e508581b77fb1 P 56abc4db7e004e008c44748e7ba48a78 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 02:36:15.858651 10499 raft_consensus.cc:482] T be900cfd63104ae89c5e508581b77fb1 P 56abc4db7e004e008c44748e7ba48a78 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 02:36:15.858728 10499 raft_consensus.cc:3037] T be900cfd63104ae89c5e508581b77fb1 P 56abc4db7e004e008c44748e7ba48a78 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:36:15.862138 10499 raft_consensus.cc:504] T be900cfd63104ae89c5e508581b77fb1 P 56abc4db7e004e008c44748e7ba48a78 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a89947c31174fd2a5b30285f3c23691" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45769 } } peers { permanent_uuid: "56abc4db7e004e008c44748e7ba48a78" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 35987 } } peers { permanent_uuid: "d3afa02e0d8841908636c0c61115dbf4" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 39029 } }
I20260812 02:36:15.862936 10499 leader_election.cc:288] T be900cfd63104ae89c5e508581b77fb1 P 56abc4db7e004e008c44748e7ba48a78 [CANDIDATE]: Term 1 election: Requested vote from peers 2a89947c31174fd2a5b30285f3c23691 (127.7.132.129:45769), d3afa02e0d8841908636c0c61115dbf4 (127.7.132.131:39029)
I20260812 02:36:15.863622 10407 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be900cfd63104ae89c5e508581b77fb1" candidate_uuid: "56abc4db7e004e008c44748e7ba48a78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3afa02e0d8841908636c0c61115dbf4"
I20260812 02:36:15.863914 10407 raft_consensus.cc:3037] T be900cfd63104ae89c5e508581b77fb1 P d3afa02e0d8841908636c0c61115dbf4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:36:15.863857 10061 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be900cfd63104ae89c5e508581b77fb1" candidate_uuid: "56abc4db7e004e008c44748e7ba48a78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a89947c31174fd2a5b30285f3c23691"
I20260812 02:36:15.864106 10061 raft_consensus.cc:3037] T be900cfd63104ae89c5e508581b77fb1 P 2a89947c31174fd2a5b30285f3c23691 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:36:15.867419 10407 raft_consensus.cc:2454] T be900cfd63104ae89c5e508581b77fb1 P d3afa02e0d8841908636c0c61115dbf4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56abc4db7e004e008c44748e7ba48a78 in term 1.
I20260812 02:36:15.867408 10061 raft_consensus.cc:2454] T be900cfd63104ae89c5e508581b77fb1 P 2a89947c31174fd2a5b30285f3c23691 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56abc4db7e004e008c44748e7ba48a78 in term 1.
I20260812 02:36:15.868005 10147 leader_election.cc:302] T be900cfd63104ae89c5e508581b77fb1 P 56abc4db7e004e008c44748e7ba48a78 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a89947c31174fd2a5b30285f3c23691, 56abc4db7e004e008c44748e7ba48a78; no voters:
I20260812 02:36:15.868374 10499 raft_consensus.cc:2781] T be900cfd63104ae89c5e508581b77fb1 P 56abc4db7e004e008c44748e7ba48a78 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:36:15.868589 10499 raft_consensus.cc:686] T be900cfd63104ae89c5e508581b77fb1 P 56abc4db7e004e008c44748e7ba48a78 [term 1 LEADER]: Becoming Leader. State: Replica: 56abc4db7e004e008c44748e7ba48a78, State: Running, Role: LEADER
I20260812 02:36:15.869058 10499 consensus_queue.cc:237] T be900cfd63104ae89c5e508581b77fb1 P 56abc4db7e004e008c44748e7ba48a78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a89947c31174fd2a5b30285f3c23691" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45769 } } peers { permanent_uuid: "56abc4db7e004e008c44748e7ba48a78" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 35987 } } peers { permanent_uuid: "d3afa02e0d8841908636c0c61115dbf4" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 39029 } }
I20260812 02:36:15.873133 9865 catalog_manager.cc:5495] T be900cfd63104ae89c5e508581b77fb1 P 56abc4db7e004e008c44748e7ba48a78 reported cstate change: term changed from 0 to 1, leader changed from <none> to 56abc4db7e004e008c44748e7ba48a78 (127.7.132.130). New cstate: current_term: 1 leader_uuid: "56abc4db7e004e008c44748e7ba48a78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a89947c31174fd2a5b30285f3c23691" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45769 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56abc4db7e004e008c44748e7ba48a78" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 35987 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d3afa02e0d8841908636c0c61115dbf4" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 39029 } health_report { overall_health: UNKNOWN } } }
I20260812 02:36:15.899108 10566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:15.891809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51144 (local address 127.7.132.130:35987)
0812 02:36:15.892667 (+ 858us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:15.892672 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:15.892695 (+ 23us) server_negotiation.cc:407] Connection header received
0812 02:36:15.892784 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:15.892788 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:15.892879 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:15.893049 (+ 170us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 02:36:15.893510 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:15.894489 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:15.896495 (+ 2006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:15.896957 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:15.897065 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:15.897612 (+ 547us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 02:36:15.898281 (+ 669us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 02:36:15.898398 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:15.898595 (+ 197us) server_negotiation.cc:299] Negotiation successful
0812 02:36:15.898680 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":742,"thread_start_us":578,"threads_started":1}
I20260812 02:36:15.916432 10486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:15.910194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:43037 (local address 127.7.132.131:39029)
0812 02:36:15.910444 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:15.910454 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:15.911136 (+ 682us) server_negotiation.cc:407] Connection header received
0812 02:36:15.911455 (+ 319us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:15.911461 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:15.911543 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:15.911644 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 02:36:15.912123 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:15.912861 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:15.914539 (+ 1678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:15.915025 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:15.915156 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:15.915625 (+ 469us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 02:36:15.915740 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 02:36:15.915842 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:15.916043 (+ 201us) server_negotiation.cc:299] Negotiation successful
0812 02:36:15.916127 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260812 02:36:15.916594 10569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:15.910173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.131:39029 (local address 127.7.132.130:43037)
0812 02:36:15.911019 (+ 846us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:15.911036 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:15.911296 (+ 260us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:15.911660 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:15.911664 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:15.911676 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 02:36:15.911976 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:15.911985 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:15.913021 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:15.913027 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:15.914376 (+ 1349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:15.914403 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:15.915180 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:15.915185 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:15.915358 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:15.915427 (+ 69us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 02:36:15.915847 (+ 420us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 02:36:15.915861 (+ 14us) client_negotiation.cc:769] Sending connection context
0812 02:36:15.915966 (+ 105us) client_negotiation.cc:240] Negotiation successful
0812 02:36:15.916179 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":687,"thread_start_us":476,"threads_started":1}
I20260812 02:36:15.927837 10407 raft_consensus.cc:1261] T be900cfd63104ae89c5e508581b77fb1 P d3afa02e0d8841908636c0c61115dbf4 [term 1 FOLLOWER]: Refusing update from remote peer 56abc4db7e004e008c44748e7ba48a78: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:36:15.927846 10061 raft_consensus.cc:1261] T be900cfd63104ae89c5e508581b77fb1 P 2a89947c31174fd2a5b30285f3c23691 [term 1 FOLLOWER]: Refusing update from remote peer 56abc4db7e004e008c44748e7ba48a78: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:36:15.928689 10542 consensus_queue.cc:1035] T be900cfd63104ae89c5e508581b77fb1 P 56abc4db7e004e008c44748e7ba48a78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a89947c31174fd2a5b30285f3c23691" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45769 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:36:15.929265 10499 consensus_queue.cc:1035] T be900cfd63104ae89c5e508581b77fb1 P 56abc4db7e004e008c44748e7ba48a78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3afa02e0d8841908636c0c61115dbf4" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 39029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:36:15.972239 10566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:15.966063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:37497 (local address 127.7.132.130:35987)
0812 02:36:15.966254 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:15.966261 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:15.966308 (+ 47us) server_negotiation.cc:407] Connection header received
0812 02:36:15.966546 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:15.966552 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:15.966634 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:15.966770 (+ 136us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 02:36:15.967231 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:15.968288 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:15.970276 (+ 1988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:15.970752 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:15.970905 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:15.971423 (+ 518us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 02:36:15.971554 (+ 131us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 02:36:15.971665 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:15.971826 (+ 161us) server_negotiation.cc:299] Negotiation successful
0812 02:36:15.971915 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260812 02:36:15.972555 10555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:15.965987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.130:35987 (local address 127.7.132.131:37497)
0812 02:36:15.966226 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:15.966250 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:15.966390 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:15.966788 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:15.966794 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:15.966806 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 02:36:15.967016 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:15.967047 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:15.968517 (+ 1470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:15.968523 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:15.969969 (+ 1446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:15.969983 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:15.970915 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:15.970921 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:15.971149 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:15.971208 (+ 59us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 02:36:15.971669 (+ 461us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 02:36:15.971685 (+ 16us) client_negotiation.cc:769] Sending connection context
0812 02:36:15.971804 (+ 119us) client_negotiation.cc:240] Negotiation successful
0812 02:36:15.972202 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
W20260812 02:36:15.973994 10535 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260812 02:36:17.619896 9865 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49418:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 02:36:17.620211 9865 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49418:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 02:36:17.624944 9865 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1a4320c66f614d04a17f19477a85c319: Sending DeleteTablet for 3 replicas of tablet be900cfd63104ae89c5e508581b77fb1
I20260812 02:36:17.626058 10380 tablet_service.cc:1514] Processing DeleteTablet for tablet be900cfd63104ae89c5e508581b77fb1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:36:17 UTC) from {username='kudu', principal='kudu/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:47862
I20260812 02:36:17.626320 10210 tablet_service.cc:1514] Processing DeleteTablet for tablet be900cfd63104ae89c5e508581b77fb1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:36:17 UTC) from {username='kudu', principal='kudu/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:51138
I20260812 02:36:17.626283 10036 tablet_service.cc:1514] Processing DeleteTablet for tablet be900cfd63104ae89c5e508581b77fb1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:36:17 UTC) from {username='kudu', principal='kudu/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:49730
I20260812 02:36:17.638590 10605 tablet_replica.cc:330] T be900cfd63104ae89c5e508581b77fb1 P d3afa02e0d8841908636c0c61115dbf4: stopping tablet replica
I20260812 02:36:17.639037 10605 raft_consensus.cc:2229] T be900cfd63104ae89c5e508581b77fb1 P d3afa02e0d8841908636c0c61115dbf4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 02:36:17.639439 10605 raft_consensus.cc:2258] T be900cfd63104ae89c5e508581b77fb1 P d3afa02e0d8841908636c0c61115dbf4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:36:17.641078 10605 ts_tablet_manager.cc:1905] T be900cfd63104ae89c5e508581b77fb1 P d3afa02e0d8841908636c0c61115dbf4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:36:17.641790 10606 tablet_replica.cc:330] T be900cfd63104ae89c5e508581b77fb1 P 56abc4db7e004e008c44748e7ba48a78: stopping tablet replica
I20260812 02:36:17.642063 10606 raft_consensus.cc:2229] T be900cfd63104ae89c5e508581b77fb1 P 56abc4db7e004e008c44748e7ba48a78 [term 1 LEADER]: Raft consensus shutting down.
I20260812 02:36:17.642652 10607 tablet_replica.cc:330] T be900cfd63104ae89c5e508581b77fb1 P 2a89947c31174fd2a5b30285f3c23691: stopping tablet replica
I20260812 02:36:17.642897 10607 raft_consensus.cc:2229] T be900cfd63104ae89c5e508581b77fb1 P 2a89947c31174fd2a5b30285f3c23691 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 02:36:17.643187 10607 raft_consensus.cc:2258] T be900cfd63104ae89c5e508581b77fb1 P 2a89947c31174fd2a5b30285f3c23691 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:36:17.643926 10606 raft_consensus.cc:2258] T be900cfd63104ae89c5e508581b77fb1 P 56abc4db7e004e008c44748e7ba48a78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:36:17.644908 10607 ts_tablet_manager.cc:1905] T be900cfd63104ae89c5e508581b77fb1 P 2a89947c31174fd2a5b30285f3c23691: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:36:17.647874 10605 ts_tablet_manager.cc:1918] T be900cfd63104ae89c5e508581b77fb1 P d3afa02e0d8841908636c0c61115dbf4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 02:36:17.648037 10605 log.cc:1196] T be900cfd63104ae89c5e508581b77fb1 P d3afa02e0d8841908636c0c61115dbf4: Deleting WAL directory at /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502156971023-7698-0/minicluster-data/ts-2/wal/wals/be900cfd63104ae89c5e508581b77fb1
I20260812 02:36:17.648550 10605 ts_tablet_manager.cc:1939] T be900cfd63104ae89c5e508581b77fb1 P d3afa02e0d8841908636c0c61115dbf4: Deleting consensus metadata
I20260812 02:36:17.649758 9850 catalog_manager.cc:4841] TS d3afa02e0d8841908636c0c61115dbf4 (127.7.132.131:39029): tablet be900cfd63104ae89c5e508581b77fb1 (table test-table [id=949a79fa70794b3991719d00e9e321e2]) successfully deleted
I20260812 02:36:17.650846 10606 ts_tablet_manager.cc:1905] T be900cfd63104ae89c5e508581b77fb1 P 56abc4db7e004e008c44748e7ba48a78: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:36:17.653385 10607 ts_tablet_manager.cc:1918] T be900cfd63104ae89c5e508581b77fb1 P 2a89947c31174fd2a5b30285f3c23691: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 02:36:17.653555 10607 log.cc:1196] T be900cfd63104ae89c5e508581b77fb1 P 2a89947c31174fd2a5b30285f3c23691: Deleting WAL directory at /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502156971023-7698-0/minicluster-data/ts-0/wal/wals/be900cfd63104ae89c5e508581b77fb1
I20260812 02:36:17.654036 10607 ts_tablet_manager.cc:1939] T be900cfd63104ae89c5e508581b77fb1 P 2a89947c31174fd2a5b30285f3c23691: Deleting consensus metadata
I20260812 02:36:17.655215 9850 catalog_manager.cc:4841] TS 2a89947c31174fd2a5b30285f3c23691 (127.7.132.129:45769): tablet be900cfd63104ae89c5e508581b77fb1 (table test-table [id=949a79fa70794b3991719d00e9e321e2]) successfully deleted
I20260812 02:36:17.657496 10606 ts_tablet_manager.cc:1918] T be900cfd63104ae89c5e508581b77fb1 P 56abc4db7e004e008c44748e7ba48a78: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 02:36:17.657667 10606 log.cc:1196] T be900cfd63104ae89c5e508581b77fb1 P 56abc4db7e004e008c44748e7ba48a78: Deleting WAL directory at /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502156971023-7698-0/minicluster-data/ts-1/wal/wals/be900cfd63104ae89c5e508581b77fb1
I20260812 02:36:17.658156 10606 ts_tablet_manager.cc:1939] T be900cfd63104ae89c5e508581b77fb1 P 56abc4db7e004e008c44748e7ba48a78: Deleting consensus metadata
I20260812 02:36:17.659528 9849 catalog_manager.cc:4841] TS 56abc4db7e004e008c44748e7ba48a78 (127.7.132.130:35987): tablet be900cfd63104ae89c5e508581b77fb1 (table test-table [id=949a79fa70794b3991719d00e9e321e2]) successfully deleted
Aug 12 02:36:17 dist-test-slave-mn4r krb5kdc[9811](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502177, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 02:36:17 dist-test-slave-mn4r krb5kdc[9811](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502177, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 02:36:17.700945 10618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:17.685285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40496 (local address 127.7.132.190:39693)
0812 02:36:17.686185 (+ 900us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:17.686190 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:17.686210 (+ 20us) server_negotiation.cc:407] Connection header received
0812 02:36:17.686266 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:17.686270 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:17.686330 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:17.686461 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:17.687896 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:17.688711 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:17.690020 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:17.690510 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:17.690664 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:17.694920 (+ 4256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:17.694926 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:17.694929 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:17.694973 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:17.697853 (+ 2880us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:17.698700 (+ 847us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:17.698708 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:17.698712 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:17.698855 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:17.699271 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:17.699276 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:17.699280 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:17.699609 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:17.699887 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:17.700274 (+ 387us) server_negotiation.cc:299] Negotiation successful
0812 02:36:17.700540 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":723,"thread_start_us":535,"threads_started":1}
I20260812 02:36:17.704818 9865 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40496:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 02:36:17.705497 9865 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 02:36:17.712932 10036 tablet_service.cc:1467] Processing CreateTablet for tablet 697399a3224e406f9d87442a212ee763 (DEFAULT_TABLE table=test-table [id=e96587dcf2b846d2ac42ada39b5eda68]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:36:17.712960 10210 tablet_service.cc:1467] Processing CreateTablet for tablet 697399a3224e406f9d87442a212ee763 (DEFAULT_TABLE table=test-table [id=e96587dcf2b846d2ac42ada39b5eda68]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:36:17.713294 10380 tablet_service.cc:1467] Processing CreateTablet for tablet 697399a3224e406f9d87442a212ee763 (DEFAULT_TABLE table=test-table [id=e96587dcf2b846d2ac42ada39b5eda68]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:36:17.713411 10036 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 697399a3224e406f9d87442a212ee763. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:36:17.713435 10210 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 697399a3224e406f9d87442a212ee763. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:36:17.713774 10380 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 697399a3224e406f9d87442a212ee763. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:36:17.719549 10619 tablet_bootstrap.cc:492] T 697399a3224e406f9d87442a212ee763 P d3afa02e0d8841908636c0c61115dbf4: Bootstrap starting.
I20260812 02:36:17.720325 10620 tablet_bootstrap.cc:492] T 697399a3224e406f9d87442a212ee763 P 56abc4db7e004e008c44748e7ba48a78: Bootstrap starting.
I20260812 02:36:17.721101 10621 tablet_bootstrap.cc:492] T 697399a3224e406f9d87442a212ee763 P 2a89947c31174fd2a5b30285f3c23691: Bootstrap starting.
I20260812 02:36:17.721500 10619 tablet_bootstrap.cc:654] T 697399a3224e406f9d87442a212ee763 P d3afa02e0d8841908636c0c61115dbf4: Neither blocks nor log segments found. Creating new log.
I20260812 02:36:17.722441 10620 tablet_bootstrap.cc:654] T 697399a3224e406f9d87442a212ee763 P 56abc4db7e004e008c44748e7ba48a78: Neither blocks nor log segments found. Creating new log.
I20260812 02:36:17.722887 10621 tablet_bootstrap.cc:654] T 697399a3224e406f9d87442a212ee763 P 2a89947c31174fd2a5b30285f3c23691: Neither blocks nor log segments found. Creating new log.
I20260812 02:36:17.725035 10620 tablet_bootstrap.cc:492] T 697399a3224e406f9d87442a212ee763 P 56abc4db7e004e008c44748e7ba48a78: No bootstrap required, opened a new log
I20260812 02:36:17.725194 10620 ts_tablet_manager.cc:1397] T 697399a3224e406f9d87442a212ee763 P 56abc4db7e004e008c44748e7ba48a78: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260812 02:36:17.725342 10621 tablet_bootstrap.cc:492] T 697399a3224e406f9d87442a212ee763 P 2a89947c31174fd2a5b30285f3c23691: No bootstrap required, opened a new log
I20260812 02:36:17.725504 10621 ts_tablet_manager.cc:1397] T 697399a3224e406f9d87442a212ee763 P 2a89947c31174fd2a5b30285f3c23691: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260812 02:36:17.726176 10619 tablet_bootstrap.cc:492] T 697399a3224e406f9d87442a212ee763 P d3afa02e0d8841908636c0c61115dbf4: No bootstrap required, opened a new log
I20260812 02:36:17.726330 10619 ts_tablet_manager.cc:1397] T 697399a3224e406f9d87442a212ee763 P d3afa02e0d8841908636c0c61115dbf4: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260812 02:36:17.726188 10620 raft_consensus.cc:348] T 697399a3224e406f9d87442a212ee763 P 56abc4db7e004e008c44748e7ba48a78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a89947c31174fd2a5b30285f3c23691" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45769 } } peers { permanent_uuid: "56abc4db7e004e008c44748e7ba48a78" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 35987 } } peers { permanent_uuid: "d3afa02e0d8841908636c0c61115dbf4" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 39029 } }
I20260812 02:36:17.726439 10620 raft_consensus.cc:374] T 697399a3224e406f9d87442a212ee763 P 56abc4db7e004e008c44748e7ba48a78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:36:17.726353 10621 raft_consensus.cc:348] T 697399a3224e406f9d87442a212ee763 P 2a89947c31174fd2a5b30285f3c23691 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a89947c31174fd2a5b30285f3c23691" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45769 } } peers { permanent_uuid: "56abc4db7e004e008c44748e7ba48a78" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 35987 } } peers { permanent_uuid: "d3afa02e0d8841908636c0c61115dbf4" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 39029 } }
I20260812 02:36:17.726557 10620 raft_consensus.cc:729] T 697399a3224e406f9d87442a212ee763 P 56abc4db7e004e008c44748e7ba48a78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56abc4db7e004e008c44748e7ba48a78, State: Initialized, Role: FOLLOWER
I20260812 02:36:17.726609 10621 raft_consensus.cc:374] T 697399a3224e406f9d87442a212ee763 P 2a89947c31174fd2a5b30285f3c23691 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:36:17.726756 10621 raft_consensus.cc:729] T 697399a3224e406f9d87442a212ee763 P 2a89947c31174fd2a5b30285f3c23691 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a89947c31174fd2a5b30285f3c23691, State: Initialized, Role: FOLLOWER
I20260812 02:36:17.726817 10620 consensus_queue.cc:260] T 697399a3224e406f9d87442a212ee763 P 56abc4db7e004e008c44748e7ba48a78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a89947c31174fd2a5b30285f3c23691" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45769 } } peers { permanent_uuid: "56abc4db7e004e008c44748e7ba48a78" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 35987 } } peers { permanent_uuid: "d3afa02e0d8841908636c0c61115dbf4" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 39029 } }
I20260812 02:36:17.727048 10621 consensus_queue.cc:260] T 697399a3224e406f9d87442a212ee763 P 2a89947c31174fd2a5b30285f3c23691 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a89947c31174fd2a5b30285f3c23691" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45769 } } peers { permanent_uuid: "56abc4db7e004e008c44748e7ba48a78" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 35987 } } peers { permanent_uuid: "d3afa02e0d8841908636c0c61115dbf4" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 39029 } }
I20260812 02:36:17.727205 10619 raft_consensus.cc:348] T 697399a3224e406f9d87442a212ee763 P d3afa02e0d8841908636c0c61115dbf4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a89947c31174fd2a5b30285f3c23691" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45769 } } peers { permanent_uuid: "56abc4db7e004e008c44748e7ba48a78" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 35987 } } peers { permanent_uuid: "d3afa02e0d8841908636c0c61115dbf4" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 39029 } }
I20260812 02:36:17.727451 10619 raft_consensus.cc:374] T 697399a3224e406f9d87442a212ee763 P d3afa02e0d8841908636c0c61115dbf4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:36:17.727572 10619 raft_consensus.cc:729] T 697399a3224e406f9d87442a212ee763 P d3afa02e0d8841908636c0c61115dbf4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3afa02e0d8841908636c0c61115dbf4, State: Initialized, Role: FOLLOWER
I20260812 02:36:17.727792 10619 consensus_queue.cc:260] T 697399a3224e406f9d87442a212ee763 P d3afa02e0d8841908636c0c61115dbf4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a89947c31174fd2a5b30285f3c23691" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45769 } } peers { permanent_uuid: "56abc4db7e004e008c44748e7ba48a78" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 35987 } } peers { permanent_uuid: "d3afa02e0d8841908636c0c61115dbf4" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 39029 } }
I20260812 02:36:17.727968 10621 ts_tablet_manager.cc:1428] T 697399a3224e406f9d87442a212ee763 P 2a89947c31174fd2a5b30285f3c23691: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260812 02:36:17.728304 10620 ts_tablet_manager.cc:1428] T 697399a3224e406f9d87442a212ee763 P 56abc4db7e004e008c44748e7ba48a78: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260812 02:36:17.728351 10619 ts_tablet_manager.cc:1428] T 697399a3224e406f9d87442a212ee763 P d3afa02e0d8841908636c0c61115dbf4: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 02:36:17.841529 10595 raft_consensus.cc:482] T 697399a3224e406f9d87442a212ee763 P 56abc4db7e004e008c44748e7ba48a78 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 02:36:17.841714 10595 raft_consensus.cc:504] T 697399a3224e406f9d87442a212ee763 P 56abc4db7e004e008c44748e7ba48a78 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a89947c31174fd2a5b30285f3c23691" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45769 } } peers { permanent_uuid: "56abc4db7e004e008c44748e7ba48a78" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 35987 } } peers { permanent_uuid: "d3afa02e0d8841908636c0c61115dbf4" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 39029 } }
I20260812 02:36:17.842557 10595 leader_election.cc:288] T 697399a3224e406f9d87442a212ee763 P 56abc4db7e004e008c44748e7ba48a78 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2a89947c31174fd2a5b30285f3c23691 (127.7.132.129:45769), d3afa02e0d8841908636c0c61115dbf4 (127.7.132.131:39029)
I20260812 02:36:17.843370 10058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "697399a3224e406f9d87442a212ee763" candidate_uuid: "56abc4db7e004e008c44748e7ba48a78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a89947c31174fd2a5b30285f3c23691" is_pre_election: true
I20260812 02:36:17.843355 10407 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "697399a3224e406f9d87442a212ee763" candidate_uuid: "56abc4db7e004e008c44748e7ba48a78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3afa02e0d8841908636c0c61115dbf4" is_pre_election: true
I20260812 02:36:17.843679 10058 raft_consensus.cc:2454] T 697399a3224e406f9d87442a212ee763 P 2a89947c31174fd2a5b30285f3c23691 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56abc4db7e004e008c44748e7ba48a78 in term 0.
I20260812 02:36:17.843706 10407 raft_consensus.cc:2454] T 697399a3224e406f9d87442a212ee763 P d3afa02e0d8841908636c0c61115dbf4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56abc4db7e004e008c44748e7ba48a78 in term 0.
I20260812 02:36:17.844290 10147 leader_election.cc:302] T 697399a3224e406f9d87442a212ee763 P 56abc4db7e004e008c44748e7ba48a78 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a89947c31174fd2a5b30285f3c23691, 56abc4db7e004e008c44748e7ba48a78; no voters:
I20260812 02:36:17.844570 10595 raft_consensus.cc:2781] T 697399a3224e406f9d87442a212ee763 P 56abc4db7e004e008c44748e7ba48a78 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 02:36:17.844686 10595 raft_consensus.cc:482] T 697399a3224e406f9d87442a212ee763 P 56abc4db7e004e008c44748e7ba48a78 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 02:36:17.844812 10595 raft_consensus.cc:3037] T 697399a3224e406f9d87442a212ee763 P 56abc4db7e004e008c44748e7ba48a78 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:36:17.847244 10595 raft_consensus.cc:504] T 697399a3224e406f9d87442a212ee763 P 56abc4db7e004e008c44748e7ba48a78 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a89947c31174fd2a5b30285f3c23691" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45769 } } peers { permanent_uuid: "56abc4db7e004e008c44748e7ba48a78" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 35987 } } peers { permanent_uuid: "d3afa02e0d8841908636c0c61115dbf4" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 39029 } }
I20260812 02:36:17.847853 10595 leader_election.cc:288] T 697399a3224e406f9d87442a212ee763 P 56abc4db7e004e008c44748e7ba48a78 [CANDIDATE]: Term 1 election: Requested vote from peers 2a89947c31174fd2a5b30285f3c23691 (127.7.132.129:45769), d3afa02e0d8841908636c0c61115dbf4 (127.7.132.131:39029)
I20260812 02:36:17.848384 10058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "697399a3224e406f9d87442a212ee763" candidate_uuid: "56abc4db7e004e008c44748e7ba48a78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a89947c31174fd2a5b30285f3c23691"
I20260812 02:36:17.848634 10058 raft_consensus.cc:3037] T 697399a3224e406f9d87442a212ee763 P 2a89947c31174fd2a5b30285f3c23691 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:36:17.848557 10407 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "697399a3224e406f9d87442a212ee763" candidate_uuid: "56abc4db7e004e008c44748e7ba48a78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3afa02e0d8841908636c0c61115dbf4"
I20260812 02:36:17.848887 10407 raft_consensus.cc:3037] T 697399a3224e406f9d87442a212ee763 P d3afa02e0d8841908636c0c61115dbf4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:36:17.850929 10058 raft_consensus.cc:2454] T 697399a3224e406f9d87442a212ee763 P 2a89947c31174fd2a5b30285f3c23691 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56abc4db7e004e008c44748e7ba48a78 in term 1.
I20260812 02:36:17.851115 10407 raft_consensus.cc:2454] T 697399a3224e406f9d87442a212ee763 P d3afa02e0d8841908636c0c61115dbf4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56abc4db7e004e008c44748e7ba48a78 in term 1.
I20260812 02:36:17.851399 10147 leader_election.cc:302] T 697399a3224e406f9d87442a212ee763 P 56abc4db7e004e008c44748e7ba48a78 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a89947c31174fd2a5b30285f3c23691, 56abc4db7e004e008c44748e7ba48a78; no voters:
I20260812 02:36:17.851718 10595 raft_consensus.cc:2781] T 697399a3224e406f9d87442a212ee763 P 56abc4db7e004e008c44748e7ba48a78 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:36:17.851879 10595 raft_consensus.cc:686] T 697399a3224e406f9d87442a212ee763 P 56abc4db7e004e008c44748e7ba48a78 [term 1 LEADER]: Becoming Leader. State: Replica: 56abc4db7e004e008c44748e7ba48a78, State: Running, Role: LEADER
I20260812 02:36:17.852208 10595 consensus_queue.cc:237] T 697399a3224e406f9d87442a212ee763 P 56abc4db7e004e008c44748e7ba48a78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a89947c31174fd2a5b30285f3c23691" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45769 } } peers { permanent_uuid: "56abc4db7e004e008c44748e7ba48a78" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 35987 } } peers { permanent_uuid: "d3afa02e0d8841908636c0c61115dbf4" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 39029 } }
I20260812 02:36:17.855132 9870 catalog_manager.cc:5495] T 697399a3224e406f9d87442a212ee763 P 56abc4db7e004e008c44748e7ba48a78 reported cstate change: term changed from 0 to 1, leader changed from <none> to 56abc4db7e004e008c44748e7ba48a78 (127.7.132.130). New cstate: current_term: 1 leader_uuid: "56abc4db7e004e008c44748e7ba48a78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a89947c31174fd2a5b30285f3c23691" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45769 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56abc4db7e004e008c44748e7ba48a78" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 35987 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d3afa02e0d8841908636c0c61115dbf4" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 39029 } health_report { overall_health: UNKNOWN } } }
I20260812 02:36:17.887291 10629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:17.881581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49534 (local address 127.7.132.130:35987)
0812 02:36:17.882281 (+ 700us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:17.882286 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:17.882306 (+ 20us) server_negotiation.cc:407] Connection header received
0812 02:36:17.882360 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:17.882363 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:17.882425 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:17.882518 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 02:36:17.883054 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:17.883841 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:17.885330 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:17.885980 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:17.886101 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:17.886614 (+ 513us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 02:36:17.886716 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 02:36:17.886789 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:17.886939 (+ 150us) server_negotiation.cc:299] Negotiation successful
0812 02:36:17.887026 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":533,"thread_start_us":373,"threads_started":1}
I20260812 02:36:17.896842 10407 raft_consensus.cc:1261] T 697399a3224e406f9d87442a212ee763 P d3afa02e0d8841908636c0c61115dbf4 [term 1 FOLLOWER]: Refusing update from remote peer 56abc4db7e004e008c44748e7ba48a78: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:36:17.896806 10058 raft_consensus.cc:1261] T 697399a3224e406f9d87442a212ee763 P 2a89947c31174fd2a5b30285f3c23691 [term 1 FOLLOWER]: Refusing update from remote peer 56abc4db7e004e008c44748e7ba48a78: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:36:17.897408 10595 consensus_queue.cc:1035] T 697399a3224e406f9d87442a212ee763 P 56abc4db7e004e008c44748e7ba48a78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a89947c31174fd2a5b30285f3c23691" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45769 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:36:17.897763 10599 consensus_queue.cc:1035] T 697399a3224e406f9d87442a212ee763 P 56abc4db7e004e008c44748e7ba48a78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3afa02e0d8841908636c0c61115dbf4" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 39029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:36:17.946779 9867 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40496:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 02:36:17.947041 9867 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40496:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 02:36:17.950212 9867 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1a4320c66f614d04a17f19477a85c319: Sending DeleteTablet for 3 replicas of tablet 697399a3224e406f9d87442a212ee763
I20260812 02:36:17.951046 10036 tablet_service.cc:1514] Processing DeleteTablet for tablet 697399a3224e406f9d87442a212ee763 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:36:17 UTC) from {username='kudu', principal='kudu/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:49730
I20260812 02:36:17.951049 10210 tablet_service.cc:1514] Processing DeleteTablet for tablet 697399a3224e406f9d87442a212ee763 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:36:17 UTC) from {username='kudu', principal='kudu/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:51138
I20260812 02:36:17.951357 10606 tablet_replica.cc:330] T 697399a3224e406f9d87442a212ee763 P 56abc4db7e004e008c44748e7ba48a78: stopping tablet replica
I20260812 02:36:17.951356 10607 tablet_replica.cc:330] T 697399a3224e406f9d87442a212ee763 P 2a89947c31174fd2a5b30285f3c23691: stopping tablet replica
I20260812 02:36:17.951403 10380 tablet_service.cc:1514] Processing DeleteTablet for tablet 697399a3224e406f9d87442a212ee763 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:36:17 UTC) from {username='kudu', principal='kudu/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:47862
I20260812 02:36:17.951578 10606 raft_consensus.cc:2229] T 697399a3224e406f9d87442a212ee763 P 56abc4db7e004e008c44748e7ba48a78 [term 1 LEADER]: Raft consensus shutting down.
I20260812 02:36:17.951577 10607 raft_consensus.cc:2229] T 697399a3224e406f9d87442a212ee763 P 2a89947c31174fd2a5b30285f3c23691 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 02:36:17.951807 10607 raft_consensus.cc:2258] T 697399a3224e406f9d87442a212ee763 P 2a89947c31174fd2a5b30285f3c23691 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:36:17.951918 10606 raft_consensus.cc:2258] T 697399a3224e406f9d87442a212ee763 P 56abc4db7e004e008c44748e7ba48a78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:36:17.952263 10605 tablet_replica.cc:330] T 697399a3224e406f9d87442a212ee763 P d3afa02e0d8841908636c0c61115dbf4: stopping tablet replica
I20260812 02:36:17.952528 10605 raft_consensus.cc:2229] T 697399a3224e406f9d87442a212ee763 P d3afa02e0d8841908636c0c61115dbf4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 02:36:17.952777 10605 raft_consensus.cc:2258] T 697399a3224e406f9d87442a212ee763 P d3afa02e0d8841908636c0c61115dbf4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:36:17.953012 10606 ts_tablet_manager.cc:1905] T 697399a3224e406f9d87442a212ee763 P 56abc4db7e004e008c44748e7ba48a78: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:36:17.953070 10607 ts_tablet_manager.cc:1905] T 697399a3224e406f9d87442a212ee763 P 2a89947c31174fd2a5b30285f3c23691: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:36:17.953925 10605 ts_tablet_manager.cc:1905] T 697399a3224e406f9d87442a212ee763 P d3afa02e0d8841908636c0c61115dbf4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:36:17.958459 9867 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40496:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260812 02:36:17.958782 10605 ts_tablet_manager.cc:1918] T 697399a3224e406f9d87442a212ee763 P d3afa02e0d8841908636c0c61115dbf4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 02:36:17.958923 10605 log.cc:1196] T 697399a3224e406f9d87442a212ee763 P d3afa02e0d8841908636c0c61115dbf4: Deleting WAL directory at /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502156971023-7698-0/minicluster-data/ts-2/wal/wals/697399a3224e406f9d87442a212ee763
W20260812 02:36:17.959281 9867 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 02:36:17.959374 10605 ts_tablet_manager.cc:1939] T 697399a3224e406f9d87442a212ee763 P d3afa02e0d8841908636c0c61115dbf4: Deleting consensus metadata
I20260812 02:36:17.959868 10606 ts_tablet_manager.cc:1918] T 697399a3224e406f9d87442a212ee763 P 56abc4db7e004e008c44748e7ba48a78: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 02:36:17.959882 10607 ts_tablet_manager.cc:1918] T 697399a3224e406f9d87442a212ee763 P 2a89947c31174fd2a5b30285f3c23691: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 02:36:17.960022 10606 log.cc:1196] T 697399a3224e406f9d87442a212ee763 P 56abc4db7e004e008c44748e7ba48a78: Deleting WAL directory at /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502156971023-7698-0/minicluster-data/ts-1/wal/wals/697399a3224e406f9d87442a212ee763
I20260812 02:36:17.960084 10607 log.cc:1196] T 697399a3224e406f9d87442a212ee763 P 2a89947c31174fd2a5b30285f3c23691: Deleting WAL directory at /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502156971023-7698-0/minicluster-data/ts-0/wal/wals/697399a3224e406f9d87442a212ee763
I20260812 02:36:17.960525 10606 ts_tablet_manager.cc:1939] T 697399a3224e406f9d87442a212ee763 P 56abc4db7e004e008c44748e7ba48a78: Deleting consensus metadata
I20260812 02:36:17.960582 9850 catalog_manager.cc:4841] TS d3afa02e0d8841908636c0c61115dbf4 (127.7.132.131:39029): tablet 697399a3224e406f9d87442a212ee763 (table test-table [id=e96587dcf2b846d2ac42ada39b5eda68]) successfully deleted
I20260812 02:36:17.960623 10607 ts_tablet_manager.cc:1939] T 697399a3224e406f9d87442a212ee763 P 2a89947c31174fd2a5b30285f3c23691: Deleting consensus metadata
I20260812 02:36:17.961553 9849 catalog_manager.cc:4841] TS 56abc4db7e004e008c44748e7ba48a78 (127.7.132.130:35987): tablet 697399a3224e406f9d87442a212ee763 (table test-table [id=e96587dcf2b846d2ac42ada39b5eda68]) successfully deleted
I20260812 02:36:17.962738 9850 catalog_manager.cc:4841] TS 2a89947c31174fd2a5b30285f3c23691 (127.7.132.129:45769): tablet 697399a3224e406f9d87442a212ee763 (table test-table [id=e96587dcf2b846d2ac42ada39b5eda68]) successfully deleted
I20260812 02:36:17.966692 10036 tablet_service.cc:1467] Processing CreateTablet for tablet f0ca1a4ac26d4b7a8deee617e70239e1 (DEFAULT_TABLE table=test-table [id=a1e3c0dc256c41e3a8fd5b1e5b502046]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:36:17.967201 10036 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f0ca1a4ac26d4b7a8deee617e70239e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:36:17.967114 10380 tablet_service.cc:1467] Processing CreateTablet for tablet f0ca1a4ac26d4b7a8deee617e70239e1 (DEFAULT_TABLE table=test-table [id=a1e3c0dc256c41e3a8fd5b1e5b502046]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:36:17.967518 10380 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f0ca1a4ac26d4b7a8deee617e70239e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:36:17.973932 10621 tablet_bootstrap.cc:492] T f0ca1a4ac26d4b7a8deee617e70239e1 P 2a89947c31174fd2a5b30285f3c23691: Bootstrap starting.
I20260812 02:36:17.973932 10619 tablet_bootstrap.cc:492] T f0ca1a4ac26d4b7a8deee617e70239e1 P d3afa02e0d8841908636c0c61115dbf4: Bootstrap starting.
I20260812 02:36:17.974390 10210 tablet_service.cc:1467] Processing CreateTablet for tablet f0ca1a4ac26d4b7a8deee617e70239e1 (DEFAULT_TABLE table=test-table [id=a1e3c0dc256c41e3a8fd5b1e5b502046]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:36:17.974766 10210 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f0ca1a4ac26d4b7a8deee617e70239e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:36:17.975781 10619 tablet_bootstrap.cc:654] T f0ca1a4ac26d4b7a8deee617e70239e1 P d3afa02e0d8841908636c0c61115dbf4: Neither blocks nor log segments found. Creating new log.
I20260812 02:36:17.975781 10621 tablet_bootstrap.cc:654] T f0ca1a4ac26d4b7a8deee617e70239e1 P 2a89947c31174fd2a5b30285f3c23691: Neither blocks nor log segments found. Creating new log.
I20260812 02:36:17.978000 10621 tablet_bootstrap.cc:492] T f0ca1a4ac26d4b7a8deee617e70239e1 P 2a89947c31174fd2a5b30285f3c23691: No bootstrap required, opened a new log
I20260812 02:36:17.978295 10621 ts_tablet_manager.cc:1397] T f0ca1a4ac26d4b7a8deee617e70239e1 P 2a89947c31174fd2a5b30285f3c23691: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.001s
I20260812 02:36:17.978777 10619 tablet_bootstrap.cc:492] T f0ca1a4ac26d4b7a8deee617e70239e1 P d3afa02e0d8841908636c0c61115dbf4: No bootstrap required, opened a new log
I20260812 02:36:17.978961 10619 ts_tablet_manager.cc:1397] T f0ca1a4ac26d4b7a8deee617e70239e1 P d3afa02e0d8841908636c0c61115dbf4: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260812 02:36:17.979141 10621 raft_consensus.cc:348] T f0ca1a4ac26d4b7a8deee617e70239e1 P 2a89947c31174fd2a5b30285f3c23691 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a89947c31174fd2a5b30285f3c23691" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45769 } } peers { permanent_uuid: "56abc4db7e004e008c44748e7ba48a78" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 35987 } } peers { permanent_uuid: "d3afa02e0d8841908636c0c61115dbf4" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 39029 } }
I20260812 02:36:17.979424 10621 raft_consensus.cc:374] T f0ca1a4ac26d4b7a8deee617e70239e1 P 2a89947c31174fd2a5b30285f3c23691 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:36:17.979516 10621 raft_consensus.cc:729] T f0ca1a4ac26d4b7a8deee617e70239e1 P 2a89947c31174fd2a5b30285f3c23691 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a89947c31174fd2a5b30285f3c23691, State: Initialized, Role: FOLLOWER
I20260812 02:36:17.979779 10621 consensus_queue.cc:260] T f0ca1a4ac26d4b7a8deee617e70239e1 P 2a89947c31174fd2a5b30285f3c23691 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a89947c31174fd2a5b30285f3c23691" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45769 } } peers { permanent_uuid: "56abc4db7e004e008c44748e7ba48a78" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 35987 } } peers { permanent_uuid: "d3afa02e0d8841908636c0c61115dbf4" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 39029 } }
I20260812 02:36:17.979809 10619 raft_consensus.cc:348] T f0ca1a4ac26d4b7a8deee617e70239e1 P d3afa02e0d8841908636c0c61115dbf4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a89947c31174fd2a5b30285f3c23691" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45769 } } peers { permanent_uuid: "56abc4db7e004e008c44748e7ba48a78" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 35987 } } peers { permanent_uuid: "d3afa02e0d8841908636c0c61115dbf4" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 39029 } }
I20260812 02:36:17.980042 10619 raft_consensus.cc:374] T f0ca1a4ac26d4b7a8deee617e70239e1 P d3afa02e0d8841908636c0c61115dbf4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:36:17.980269 10619 raft_consensus.cc:729] T f0ca1a4ac26d4b7a8deee617e70239e1 P d3afa02e0d8841908636c0c61115dbf4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3afa02e0d8841908636c0c61115dbf4, State: Initialized, Role: FOLLOWER
I20260812 02:36:17.980444 10621 ts_tablet_manager.cc:1428] T f0ca1a4ac26d4b7a8deee617e70239e1 P 2a89947c31174fd2a5b30285f3c23691: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260812 02:36:17.980441 10620 tablet_bootstrap.cc:492] T f0ca1a4ac26d4b7a8deee617e70239e1 P 56abc4db7e004e008c44748e7ba48a78: Bootstrap starting.
I20260812 02:36:17.980589 10619 consensus_queue.cc:260] T f0ca1a4ac26d4b7a8deee617e70239e1 P d3afa02e0d8841908636c0c61115dbf4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a89947c31174fd2a5b30285f3c23691" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45769 } } peers { permanent_uuid: "56abc4db7e004e008c44748e7ba48a78" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 35987 } } peers { permanent_uuid: "d3afa02e0d8841908636c0c61115dbf4" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 39029 } }
I20260812 02:36:17.982162 10619 ts_tablet_manager.cc:1428] T f0ca1a4ac26d4b7a8deee617e70239e1 P d3afa02e0d8841908636c0c61115dbf4: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 02:36:17.982367 10620 tablet_bootstrap.cc:654] T f0ca1a4ac26d4b7a8deee617e70239e1 P 56abc4db7e004e008c44748e7ba48a78: Neither blocks nor log segments found. Creating new log.
I20260812 02:36:17.985677 10620 tablet_bootstrap.cc:492] T f0ca1a4ac26d4b7a8deee617e70239e1 P 56abc4db7e004e008c44748e7ba48a78: No bootstrap required, opened a new log
I20260812 02:36:17.985888 10620 ts_tablet_manager.cc:1397] T f0ca1a4ac26d4b7a8deee617e70239e1 P 56abc4db7e004e008c44748e7ba48a78: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.002s
I20260812 02:36:17.986639 10620 raft_consensus.cc:348] T f0ca1a4ac26d4b7a8deee617e70239e1 P 56abc4db7e004e008c44748e7ba48a78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a89947c31174fd2a5b30285f3c23691" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45769 } } peers { permanent_uuid: "56abc4db7e004e008c44748e7ba48a78" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 35987 } } peers { permanent_uuid: "d3afa02e0d8841908636c0c61115dbf4" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 39029 } }
I20260812 02:36:17.986829 10620 raft_consensus.cc:374] T f0ca1a4ac26d4b7a8deee617e70239e1 P 56abc4db7e004e008c44748e7ba48a78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:36:17.986889 10620 raft_consensus.cc:729] T f0ca1a4ac26d4b7a8deee617e70239e1 P 56abc4db7e004e008c44748e7ba48a78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56abc4db7e004e008c44748e7ba48a78, State: Initialized, Role: FOLLOWER
I20260812 02:36:17.987068 10620 consensus_queue.cc:260] T f0ca1a4ac26d4b7a8deee617e70239e1 P 56abc4db7e004e008c44748e7ba48a78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a89947c31174fd2a5b30285f3c23691" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45769 } } peers { permanent_uuid: "56abc4db7e004e008c44748e7ba48a78" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 35987 } } peers { permanent_uuid: "d3afa02e0d8841908636c0c61115dbf4" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 39029 } }
I20260812 02:36:17.987533 10620 ts_tablet_manager.cc:1428] T f0ca1a4ac26d4b7a8deee617e70239e1 P 56abc4db7e004e008c44748e7ba48a78: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260812 02:36:18.009248 10626 raft_consensus.cc:482] T f0ca1a4ac26d4b7a8deee617e70239e1 P 2a89947c31174fd2a5b30285f3c23691 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 02:36:18.009477 10626 raft_consensus.cc:504] T f0ca1a4ac26d4b7a8deee617e70239e1 P 2a89947c31174fd2a5b30285f3c23691 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a89947c31174fd2a5b30285f3c23691" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45769 } } peers { permanent_uuid: "56abc4db7e004e008c44748e7ba48a78" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 35987 } } peers { permanent_uuid: "d3afa02e0d8841908636c0c61115dbf4" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 39029 } }
I20260812 02:36:18.010301 10626 leader_election.cc:288] T f0ca1a4ac26d4b7a8deee617e70239e1 P 2a89947c31174fd2a5b30285f3c23691 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 56abc4db7e004e008c44748e7ba48a78 (127.7.132.130:35987), d3afa02e0d8841908636c0c61115dbf4 (127.7.132.131:39029)
I20260812 02:36:18.010942 10234 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0ca1a4ac26d4b7a8deee617e70239e1" candidate_uuid: "2a89947c31174fd2a5b30285f3c23691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56abc4db7e004e008c44748e7ba48a78" is_pre_election: true
I20260812 02:36:18.011001 10407 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0ca1a4ac26d4b7a8deee617e70239e1" candidate_uuid: "2a89947c31174fd2a5b30285f3c23691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3afa02e0d8841908636c0c61115dbf4" is_pre_election: true
I20260812 02:36:18.011287 10234 raft_consensus.cc:2454] T f0ca1a4ac26d4b7a8deee617e70239e1 P 56abc4db7e004e008c44748e7ba48a78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a89947c31174fd2a5b30285f3c23691 in term 0.
I20260812 02:36:18.011457 10407 raft_consensus.cc:2454] T f0ca1a4ac26d4b7a8deee617e70239e1 P d3afa02e0d8841908636c0c61115dbf4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a89947c31174fd2a5b30285f3c23691 in term 0.
I20260812 02:36:18.011878 9974 leader_election.cc:302] T f0ca1a4ac26d4b7a8deee617e70239e1 P 2a89947c31174fd2a5b30285f3c23691 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a89947c31174fd2a5b30285f3c23691, 56abc4db7e004e008c44748e7ba48a78; no voters:
I20260812 02:36:18.012241 10626 raft_consensus.cc:2781] T f0ca1a4ac26d4b7a8deee617e70239e1 P 2a89947c31174fd2a5b30285f3c23691 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 02:36:18.012369 10626 raft_consensus.cc:482] T f0ca1a4ac26d4b7a8deee617e70239e1 P 2a89947c31174fd2a5b30285f3c23691 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 02:36:18.012491 10626 raft_consensus.cc:3037] T f0ca1a4ac26d4b7a8deee617e70239e1 P 2a89947c31174fd2a5b30285f3c23691 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:36:18.015096 10626 raft_consensus.cc:504] T f0ca1a4ac26d4b7a8deee617e70239e1 P 2a89947c31174fd2a5b30285f3c23691 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a89947c31174fd2a5b30285f3c23691" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45769 } } peers { permanent_uuid: "56abc4db7e004e008c44748e7ba48a78" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 35987 } } peers { permanent_uuid: "d3afa02e0d8841908636c0c61115dbf4" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 39029 } }
I20260812 02:36:18.015707 10626 leader_election.cc:288] T f0ca1a4ac26d4b7a8deee617e70239e1 P 2a89947c31174fd2a5b30285f3c23691 [CANDIDATE]: Term 1 election: Requested vote from peers 56abc4db7e004e008c44748e7ba48a78 (127.7.132.130:35987), d3afa02e0d8841908636c0c61115dbf4 (127.7.132.131:39029)
I20260812 02:36:18.016304 10407 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0ca1a4ac26d4b7a8deee617e70239e1" candidate_uuid: "2a89947c31174fd2a5b30285f3c23691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3afa02e0d8841908636c0c61115dbf4"
I20260812 02:36:18.016316 10234 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0ca1a4ac26d4b7a8deee617e70239e1" candidate_uuid: "2a89947c31174fd2a5b30285f3c23691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56abc4db7e004e008c44748e7ba48a78"
I20260812 02:36:18.016551 10407 raft_consensus.cc:3037] T f0ca1a4ac26d4b7a8deee617e70239e1 P d3afa02e0d8841908636c0c61115dbf4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:36:18.016613 10234 raft_consensus.cc:3037] T f0ca1a4ac26d4b7a8deee617e70239e1 P 56abc4db7e004e008c44748e7ba48a78 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:36:18.018782 10407 raft_consensus.cc:2454] T f0ca1a4ac26d4b7a8deee617e70239e1 P d3afa02e0d8841908636c0c61115dbf4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a89947c31174fd2a5b30285f3c23691 in term 1.
I20260812 02:36:18.019174 10234 raft_consensus.cc:2454] T f0ca1a4ac26d4b7a8deee617e70239e1 P 56abc4db7e004e008c44748e7ba48a78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a89947c31174fd2a5b30285f3c23691 in term 1.
I20260812 02:36:18.019364 9975 leader_election.cc:302] T f0ca1a4ac26d4b7a8deee617e70239e1 P 2a89947c31174fd2a5b30285f3c23691 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a89947c31174fd2a5b30285f3c23691, d3afa02e0d8841908636c0c61115dbf4; no voters:
I20260812 02:36:18.019791 10626 raft_consensus.cc:2781] T f0ca1a4ac26d4b7a8deee617e70239e1 P 2a89947c31174fd2a5b30285f3c23691 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:36:18.020270 10626 raft_consensus.cc:686] T f0ca1a4ac26d4b7a8deee617e70239e1 P 2a89947c31174fd2a5b30285f3c23691 [term 1 LEADER]: Becoming Leader. State: Replica: 2a89947c31174fd2a5b30285f3c23691, State: Running, Role: LEADER
I20260812 02:36:18.020606 10626 consensus_queue.cc:237] T f0ca1a4ac26d4b7a8deee617e70239e1 P 2a89947c31174fd2a5b30285f3c23691 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a89947c31174fd2a5b30285f3c23691" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45769 } } peers { permanent_uuid: "56abc4db7e004e008c44748e7ba48a78" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 35987 } } peers { permanent_uuid: "d3afa02e0d8841908636c0c61115dbf4" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 39029 } }
I20260812 02:36:18.023658 9867 catalog_manager.cc:5495] T f0ca1a4ac26d4b7a8deee617e70239e1 P 2a89947c31174fd2a5b30285f3c23691 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2a89947c31174fd2a5b30285f3c23691 (127.7.132.129). New cstate: current_term: 1 leader_uuid: "2a89947c31174fd2a5b30285f3c23691" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a89947c31174fd2a5b30285f3c23691" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45769 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56abc4db7e004e008c44748e7ba48a78" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 35987 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d3afa02e0d8841908636c0c61115dbf4" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 39029 } health_report { overall_health: UNKNOWN } } }
I20260812 02:36:18.057335 10640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:18.051682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42716 (local address 127.7.132.129:45769)
0812 02:36:18.052469 (+ 787us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:18.052474 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:18.052492 (+ 18us) server_negotiation.cc:407] Connection header received
0812 02:36:18.052544 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:18.052547 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:18.052605 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:18.052722 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 02:36:18.053159 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:18.053950 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:18.055454 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:18.055883 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:18.056028 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:18.056511 (+ 483us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 02:36:18.056629 (+ 118us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 02:36:18.056734 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:18.056889 (+ 155us) server_negotiation.cc:299] Negotiation successful
0812 02:36:18.056970 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":666,"thread_start_us":486,"threads_started":1}
I20260812 02:36:18.067780 10407 raft_consensus.cc:1261] T f0ca1a4ac26d4b7a8deee617e70239e1 P d3afa02e0d8841908636c0c61115dbf4 [term 1 FOLLOWER]: Refusing update from remote peer 2a89947c31174fd2a5b30285f3c23691: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:36:18.068023 10234 raft_consensus.cc:1261] T f0ca1a4ac26d4b7a8deee617e70239e1 P 56abc4db7e004e008c44748e7ba48a78 [term 1 FOLLOWER]: Refusing update from remote peer 2a89947c31174fd2a5b30285f3c23691: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:36:18.068509 10626 consensus_queue.cc:1035] T f0ca1a4ac26d4b7a8deee617e70239e1 P 2a89947c31174fd2a5b30285f3c23691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3afa02e0d8841908636c0c61115dbf4" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 39029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:36:18.068814 10638 consensus_queue.cc:1035] T f0ca1a4ac26d4b7a8deee617e70239e1 P 2a89947c31174fd2a5b30285f3c23691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56abc4db7e004e008c44748e7ba48a78" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 35987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 02:36:18.095650 10583 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260812 02:36:19.699748 9867 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40496:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 02:36:19.699963 9867 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40496:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 02:36:19.704061 9867 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1a4320c66f614d04a17f19477a85c319: Sending DeleteTablet for 3 replicas of tablet f0ca1a4ac26d4b7a8deee617e70239e1
I20260812 02:36:19.704846 10036 tablet_service.cc:1514] Processing DeleteTablet for tablet f0ca1a4ac26d4b7a8deee617e70239e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:36:19 UTC) from {username='kudu', principal='kudu/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:49730
I20260812 02:36:19.704916 10210 tablet_service.cc:1514] Processing DeleteTablet for tablet f0ca1a4ac26d4b7a8deee617e70239e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:36:19 UTC) from {username='kudu', principal='kudu/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:51138
I20260812 02:36:19.705049 10381 tablet_service.cc:1514] Processing DeleteTablet for tablet f0ca1a4ac26d4b7a8deee617e70239e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:36:19 UTC) from {username='kudu', principal='kudu/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:47862
I20260812 02:36:19.705760 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 9952
I20260812 02:36:19.713912 10675 tablet_replica.cc:330] T f0ca1a4ac26d4b7a8deee617e70239e1 P 56abc4db7e004e008c44748e7ba48a78: stopping tablet replica
I20260812 02:36:19.714221 10675 raft_consensus.cc:2229] T f0ca1a4ac26d4b7a8deee617e70239e1 P 56abc4db7e004e008c44748e7ba48a78 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 02:36:19.714624 10675 raft_consensus.cc:2258] T f0ca1a4ac26d4b7a8deee617e70239e1 P 56abc4db7e004e008c44748e7ba48a78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:36:19.715291 10676 tablet_replica.cc:330] T f0ca1a4ac26d4b7a8deee617e70239e1 P d3afa02e0d8841908636c0c61115dbf4: stopping tablet replica
I20260812 02:36:19.715507 10676 raft_consensus.cc:2229] T f0ca1a4ac26d4b7a8deee617e70239e1 P d3afa02e0d8841908636c0c61115dbf4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 02:36:19.715767 10676 raft_consensus.cc:2258] T f0ca1a4ac26d4b7a8deee617e70239e1 P d3afa02e0d8841908636c0c61115dbf4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:36:19.715876 10675 ts_tablet_manager.cc:1905] T f0ca1a4ac26d4b7a8deee617e70239e1 P 56abc4db7e004e008c44748e7ba48a78: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:36:19.717502 10676 ts_tablet_manager.cc:1905] T f0ca1a4ac26d4b7a8deee617e70239e1 P d3afa02e0d8841908636c0c61115dbf4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:36:19.719071 10674 tablet_replica.cc:330] T f0ca1a4ac26d4b7a8deee617e70239e1 P 2a89947c31174fd2a5b30285f3c23691: stopping tablet replica
I20260812 02:36:19.719280 10674 raft_consensus.cc:2229] T f0ca1a4ac26d4b7a8deee617e70239e1 P 2a89947c31174fd2a5b30285f3c23691 [term 1 LEADER]: Raft consensus shutting down.
I20260812 02:36:19.719764 10674 raft_consensus.cc:2258] T f0ca1a4ac26d4b7a8deee617e70239e1 P 2a89947c31174fd2a5b30285f3c23691 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:36:19.720754 10674 ts_tablet_manager.cc:1905] T f0ca1a4ac26d4b7a8deee617e70239e1 P 2a89947c31174fd2a5b30285f3c23691: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:36:19.722730 10675 ts_tablet_manager.cc:1918] T f0ca1a4ac26d4b7a8deee617e70239e1 P 56abc4db7e004e008c44748e7ba48a78: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 02:36:19.722886 10675 log.cc:1196] T f0ca1a4ac26d4b7a8deee617e70239e1 P 56abc4db7e004e008c44748e7ba48a78: Deleting WAL directory at /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502156971023-7698-0/minicluster-data/ts-1/wal/wals/f0ca1a4ac26d4b7a8deee617e70239e1
I20260812 02:36:19.723428 10675 ts_tablet_manager.cc:1939] T f0ca1a4ac26d4b7a8deee617e70239e1 P 56abc4db7e004e008c44748e7ba48a78: Deleting consensus metadata
I20260812 02:36:19.724514 9849 catalog_manager.cc:4841] TS 56abc4db7e004e008c44748e7ba48a78 (127.7.132.130:35987): tablet f0ca1a4ac26d4b7a8deee617e70239e1 (table test-table [id=a1e3c0dc256c41e3a8fd5b1e5b502046]) successfully deleted
I20260812 02:36:19.725142 10676 ts_tablet_manager.cc:1918] T f0ca1a4ac26d4b7a8deee617e70239e1 P d3afa02e0d8841908636c0c61115dbf4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 02:36:19.725287 10676 log.cc:1196] T f0ca1a4ac26d4b7a8deee617e70239e1 P d3afa02e0d8841908636c0c61115dbf4: Deleting WAL directory at /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502156971023-7698-0/minicluster-data/ts-2/wal/wals/f0ca1a4ac26d4b7a8deee617e70239e1
I20260812 02:36:19.725736 10676 ts_tablet_manager.cc:1939] T f0ca1a4ac26d4b7a8deee617e70239e1 P d3afa02e0d8841908636c0c61115dbf4: Deleting consensus metadata
I20260812 02:36:19.726933 9850 catalog_manager.cc:4841] TS d3afa02e0d8841908636c0c61115dbf4 (127.7.132.131:39029): tablet f0ca1a4ac26d4b7a8deee617e70239e1 (table test-table [id=a1e3c0dc256c41e3a8fd5b1e5b502046]) successfully deleted
I20260812 02:36:19.727474 10674 ts_tablet_manager.cc:1918] T f0ca1a4ac26d4b7a8deee617e70239e1 P 2a89947c31174fd2a5b30285f3c23691: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 02:36:19.727581 10674 log.cc:1196] T f0ca1a4ac26d4b7a8deee617e70239e1 P 2a89947c31174fd2a5b30285f3c23691: Deleting WAL directory at /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786502156971023-7698-0/minicluster-data/ts-0/wal/wals/f0ca1a4ac26d4b7a8deee617e70239e1
I20260812 02:36:19.728050 10674 ts_tablet_manager.cc:1939] T f0ca1a4ac26d4b7a8deee617e70239e1 P 2a89947c31174fd2a5b30285f3c23691: Deleting consensus metadata
I20260812 02:36:19.729094 9850 catalog_manager.cc:4841] TS 2a89947c31174fd2a5b30285f3c23691 (127.7.132.129:45769): tablet f0ca1a4ac26d4b7a8deee617e70239e1 (table test-table [id=a1e3c0dc256c41e3a8fd5b1e5b502046]) successfully deleted
Aug 12 02:36:19 dist-test-slave-mn4r krb5kdc[9811](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502177, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.132.129@KRBTEST.COM
I20260812 02:36:19.735648 10677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:19.707572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42724 (local address 127.7.132.129:45769)
0812 02:36:19.714576 (+ 7004us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:19.714584 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:19.714613 (+ 29us) server_negotiation.cc:407] Connection header received
0812 02:36:19.714678 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:19.714683 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:19.714772 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:19.714893 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:19.716587 (+ 1694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:19.717779 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:19.719737 (+ 1958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:19.720747 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:19.720871 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:19.730219 (+ 9348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:19.730235 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:19.730252 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:19.730294 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:19.733086 (+ 2792us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:19.733744 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:19.733751 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:19.733753 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:19.733888 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:19.734342 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:19.734346 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:19.734348 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:19.734737 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:19.734961 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:19.735190 (+ 229us) server_negotiation.cc:299] Negotiation successful
0812 02:36:19.735367 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6860,"thread_start_us":6704,"threads_started":1}
W20260812 02:36:19.736359 10112 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42724
W20260812 02:36:19.737075 7698 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 9952: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 02:36:19.737275 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 9952
I20260812 02:36:19.768271 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 10121
Aug 12 02:36:19 dist-test-slave-mn4r krb5kdc[9811](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502177, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.132.130@KRBTEST.COM
I20260812 02:36:19.784443 10679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:19.769080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49548 (local address 127.7.132.130:35987)
0812 02:36:19.769661 (+ 581us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:19.769666 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:19.769680 (+ 14us) server_negotiation.cc:407] Connection header received
0812 02:36:19.769758 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:19.769762 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:19.769822 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:19.769974 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:19.771370 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:19.772207 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:19.773459 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:19.773896 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:19.774065 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:19.778643 (+ 4578us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:19.778659 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:19.778674 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:19.778716 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:19.781463 (+ 2747us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:19.782204 (+ 741us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:19.782212 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:19.782214 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:19.782358 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:19.782759 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:19.782764 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:19.782766 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:19.783149 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:19.783504 (+ 355us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:19.783835 (+ 331us) server_negotiation.cc:299] Negotiation successful
0812 02:36:19.784076 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":456,"thread_start_us":283,"threads_started":1}
W20260812 02:36:19.785247 10286 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49548
W20260812 02:36:19.785894 7698 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 10121: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 02:36:19.786057 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 10121
I20260812 02:36:19.814705 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 10297
Aug 12 02:36:19 dist-test-slave-mn4r krb5kdc[9811](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502177, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.132.131@KRBTEST.COM
I20260812 02:36:19.831621 10680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:19.815555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54922 (local address 127.7.132.131:39029)
0812 02:36:19.816161 (+ 606us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:19.816167 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:19.816185 (+ 18us) server_negotiation.cc:407] Connection header received
0812 02:36:19.816244 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:19.816248 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:19.816310 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:19.816412 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:19.817843 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:19.818748 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:19.820056 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:19.820605 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:19.820719 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:19.825313 (+ 4594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:19.825330 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:19.825347 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:19.825407 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:19.828639 (+ 3232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:19.829330 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:19.829339 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:19.829342 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:19.829549 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:19.829902 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:19.829907 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:19.829911 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:19.830386 (+ 475us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:19.830671 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:19.830971 (+ 300us) server_negotiation.cc:299] Negotiation successful
0812 02:36:19.831167 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":480,"thread_start_us":298,"threads_started":1}
W20260812 02:36:19.832377 10462 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54922
W20260812 02:36:19.833093 7698 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 10297: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 02:36:19.833304 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 10297
I20260812 02:36:19.865480 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 9829
I20260812 02:36:19.866945 9916 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:36:19.986855 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 9829
2026-08-12T02:36:20Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (8988 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502156971023-7698-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 02:36:20 dist-test-slave-mn4r krb5kdc[10685](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 02:36:20 dist-test-slave-mn4r krb5kdc[10685](info): set up 2 sockets
Aug 12 02:36:20 dist-test-slave-mn4r krb5kdc[10685](info): commencing operation
krb5kdc: starting...
W20260812 02:36:21.198597 7698 mini_kdc.cc:121] Time spent starting KDC: real 1.161s user 0.002s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502156971023-7698-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502156971023-7698-0/krb5kdc/test-user.keytab.
Aug 12 02:36:21 dist-test-slave-mn4r krb5kdc[10685](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502181, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T02:36:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T02:36:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.132.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
I20260812 02:36:21.422843 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502156971023-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502156971023-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502156971023-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:36655
--webserver_interface=127.7.132.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:36441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.132.190:36655
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:36:21.760392 10702 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:36:21.760696 10702 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:36:21.760874 10702 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:36:21.768139 10702 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:36:21.768241 10702 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:36:21.768350 10702 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:36:21.768474 10702 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:36:21.768541 10702 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:36:21.779134 10702 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:36441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502156971023-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502156971023-7698-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.132.190:36655
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:36655
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.132.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502156971023-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:36:21.780541 10702 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:36:21.781975 10702 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:36:21.794778 10707 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:36:21.794835 10710 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:36:21.794735 10708 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:36:21.795732 10702 server_base.cc:1034] running on GCE node
I20260812 02:36:21.796551 10702 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:36:21.798027 10702 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:36:21.799292 10702 hybrid_clock.cc:648] HybridClock initialized: now 1786502181799255 us; error 37 us; skew 500 ppm
I20260812 02:36:21.799664 10702 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:36:21 dist-test-slave-mn4r krb5kdc[10685](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502181, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:36:21.802986 10702 init.cc:377] Logged in from keytab as kudu/127.7.132.190@KRBTEST.COM (short username kudu)
I20260812 02:36:21.804484 10702 webserver.cc:460] Webserver started at http://127.7.132.190:35483/ using document root <none> and password file <none>
I20260812 02:36:21.805197 10702 fs_manager.cc:359] Metadata directory not provided
I20260812 02:36:21.805346 10702 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:36:21.805683 10702 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:36:21.808841 10702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502156971023-7698-0/minicluster-data/master-0/data/instance:
uuid: "ca67c71a55e5494bb0039dae1b5715f8"
format_stamp: "Formatted at 2026-08-12 02:36:21 on dist-test-slave-mn4r"
server_key: "1be7c23746518f2565dd98c5e526cde8"
server_key_iv: "41ae082f0372ccaaa52d32977fd25159"
server_key_version: "encryptionkey@0"
I20260812 02:36:21.809651 10702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502156971023-7698-0/minicluster-data/master-0/wal/instance:
uuid: "ca67c71a55e5494bb0039dae1b5715f8"
format_stamp: "Formatted at 2026-08-12 02:36:21 on dist-test-slave-mn4r"
server_key: "1be7c23746518f2565dd98c5e526cde8"
server_key_iv: "41ae082f0372ccaaa52d32977fd25159"
server_key_version: "encryptionkey@0"
I20260812 02:36:21.814994 10702 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260812 02:36:21.818497 10717 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:36:21.819829 10702 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260812 02:36:21.820031 10702 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502156971023-7698-0/minicluster-data/master-0/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502156971023-7698-0/minicluster-data/master-0/wal
uuid: "ca67c71a55e5494bb0039dae1b5715f8"
format_stamp: "Formatted at 2026-08-12 02:36:21 on dist-test-slave-mn4r"
server_key: "1be7c23746518f2565dd98c5e526cde8"
server_key_iv: "41ae082f0372ccaaa52d32977fd25159"
server_key_version: "encryptionkey@0"
I20260812 02:36:21.820246 10702 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502156971023-7698-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502156971023-7698-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502156971023-7698-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:36:21.840711 10702 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:36:21.845048 10702 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:36:21.845351 10702 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:36:21.864921 10702 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.190:36655
I20260812 02:36:21.864920 10777 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.190:36655 every 8 connection(s)
I20260812 02:36:21.866436 10702 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
I20260812 02:36:21.870368 10778 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:36:21.875360 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 10702
I20260812 02:36:21.875628 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502156971023-7698-0/minicluster-data/master-0/wal/instance
I20260812 02:36:21.876281 7698 external_mini_cluster.cc:1428] Setting key 31cde81d6c7ba50f4ff7b2efcf0ce7c2
I20260812 02:36:21.882205 10778 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca67c71a55e5494bb0039dae1b5715f8: Bootstrap starting.
I20260812 02:36:21.885967 10778 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ca67c71a55e5494bb0039dae1b5715f8: Neither blocks nor log segments found. Creating new log.
I20260812 02:36:21.887276 10778 log.cc:824] T 00000000000000000000000000000000 P ca67c71a55e5494bb0039dae1b5715f8: Log is configured to *not* fsync() on all Append() calls
I20260812 02:36:21.890156 10778 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca67c71a55e5494bb0039dae1b5715f8: No bootstrap required, opened a new log
Aug 12 02:36:21 dist-test-slave-mn4r krb5kdc[10685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502181, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 02:36:21.897938 10778 raft_consensus.cc:348] T 00000000000000000000000000000000 P ca67c71a55e5494bb0039dae1b5715f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca67c71a55e5494bb0039dae1b5715f8" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 36655 } }
I20260812 02:36:21.898419 10778 raft_consensus.cc:374] T 00000000000000000000000000000000 P ca67c71a55e5494bb0039dae1b5715f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:36:21.898547 10778 raft_consensus.cc:729] T 00000000000000000000000000000000 P ca67c71a55e5494bb0039dae1b5715f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca67c71a55e5494bb0039dae1b5715f8, State: Initialized, Role: FOLLOWER
I20260812 02:36:21.899286 10778 consensus_queue.cc:260] T 00000000000000000000000000000000 P ca67c71a55e5494bb0039dae1b5715f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca67c71a55e5494bb0039dae1b5715f8" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 36655 } }
I20260812 02:36:21.899602 10778 raft_consensus.cc:388] T 00000000000000000000000000000000 P ca67c71a55e5494bb0039dae1b5715f8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 02:36:21.899755 10778 raft_consensus.cc:482] T 00000000000000000000000000000000 P ca67c71a55e5494bb0039dae1b5715f8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 02:36:21.899955 10778 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ca67c71a55e5494bb0039dae1b5715f8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:36:21.900779 10781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:21.878500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33922 (local address 127.7.132.190:36655)
0812 02:36:21.879383 (+ 883us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:21.879400 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:21.879466 (+ 66us) server_negotiation.cc:407] Connection header received
0812 02:36:21.881844 (+ 2378us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:21.881872 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:21.882425 (+ 553us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:21.882895 (+ 470us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:21.884173 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:21.886221 (+ 2048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:21.887548 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:21.888711 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:21.888919 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:21.893036 (+ 4117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:21.893045 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:21.893060 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:21.893102 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:21.896357 (+ 3255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:21.897085 (+ 728us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:21.897101 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:21.897117 (+ 16us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:21.897306 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:21.897703 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:21.897711 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:21.897714 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:21.898297 (+ 583us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:21.898666 (+ 369us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:21.899120 (+ 454us) server_negotiation.cc:299] Negotiation successful
0812 02:36:21.899535 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":624,"thread_start_us":390,"threads_started":1}
I20260812 02:36:21.902670 10778 raft_consensus.cc:504] T 00000000000000000000000000000000 P ca67c71a55e5494bb0039dae1b5715f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca67c71a55e5494bb0039dae1b5715f8" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 36655 } }
I20260812 02:36:21.903249 10778 leader_election.cc:302] T 00000000000000000000000000000000 P ca67c71a55e5494bb0039dae1b5715f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ca67c71a55e5494bb0039dae1b5715f8; no voters:
I20260812 02:36:21.903949 10778 leader_election.cc:288] T 00000000000000000000000000000000 P ca67c71a55e5494bb0039dae1b5715f8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 02:36:21.904067 10783 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ca67c71a55e5494bb0039dae1b5715f8 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:36:21.904778 10783 raft_consensus.cc:686] T 00000000000000000000000000000000 P ca67c71a55e5494bb0039dae1b5715f8 [term 1 LEADER]: Becoming Leader. State: Replica: ca67c71a55e5494bb0039dae1b5715f8, State: Running, Role: LEADER
I20260812 02:36:21.905328 10783 consensus_queue.cc:237] T 00000000000000000000000000000000 P ca67c71a55e5494bb0039dae1b5715f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca67c71a55e5494bb0039dae1b5715f8" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 36655 } }
I20260812 02:36:21.906291 10778 sys_catalog.cc:564] T 00000000000000000000000000000000 P ca67c71a55e5494bb0039dae1b5715f8 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:36:21.908131 10784 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca67c71a55e5494bb0039dae1b5715f8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ca67c71a55e5494bb0039dae1b5715f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca67c71a55e5494bb0039dae1b5715f8" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 36655 } } }
I20260812 02:36:21.908507 10784 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca67c71a55e5494bb0039dae1b5715f8 [sys.catalog]: This master's current role is: LEADER
I20260812 02:36:21.913540 10785 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca67c71a55e5494bb0039dae1b5715f8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ca67c71a55e5494bb0039dae1b5715f8. Latest consensus state: current_term: 1 leader_uuid: "ca67c71a55e5494bb0039dae1b5715f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca67c71a55e5494bb0039dae1b5715f8" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 36655 } } }
I20260812 02:36:21.913987 10785 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca67c71a55e5494bb0039dae1b5715f8 [sys.catalog]: This master's current role is: LEADER
W20260812 02:36:21.917800 10804 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ca67c71a55e5494bb0039dae1b5715f8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 02:36:21.917924 10804 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 02:36:21.918031 10792 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:36:21.920149 10792 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 02:36:21.929093 10792 catalog_manager.cc:1347] Generated new cluster ID: 376cb73e836b42bfa888dc82f19bb9be
I20260812 02:36:21.929204 10792 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 02:36:21.954480 10792 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 02:36:21.955801 10792 catalog_manager.cc:1504] Loading token signing keys...
I20260812 02:36:21.969905 10792 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ca67c71a55e5494bb0039dae1b5715f8: Generated new TSK 0
I20260812 02:36:21.970774 10792 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.132.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.129.keytab.
Entry for principal kudu/127.7.132.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.129.keytab.
I20260812 02:36:22.033527 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502156971023-7698-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502156971023-7698-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502156971023-7698-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502156971023-7698-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.129:0
--local_ip_for_outbound_sockets=127.7.132.129
--webserver_interface=127.7.132.129
--webserver_port=0
--tserver_master_addrs=127.7.132.190:36655
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.129.keytab
--principal=kudu/127.7.132.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:36441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:36:22.377108 10809 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:36:22.377414 10809 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:36:22.377519 10809 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:36:22.385533 10809 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:36:22.385645 10809 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:36:22.385758 10809 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.129
I20260812 02:36:22.395634 10809 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:36441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502156971023-7698-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502156971023-7698-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.129.keytab
--principal=kudu/127.7.132.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502156971023-7698-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.132.129
--webserver_port=0
--tserver_master_addrs=127.7.132.190:36655
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.129
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502156971023-7698-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:36:22.396926 10809 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:36:22.398403 10809 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:36:22.413232 10815 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:36:22.413264 10814 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:36:22.413317 10818 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:36:22.414518 10809 server_base.cc:1034] running on GCE node
I20260812 02:36:22.415405 10809 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:36:22.416389 10809 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:36:22.417623 10809 hybrid_clock.cc:648] HybridClock initialized: now 1786502182417572 us; error 43 us; skew 500 ppm
I20260812 02:36:22.417969 10809 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:36:22 dist-test-slave-mn4r krb5kdc[10685](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502182, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:36:22.421490 10809 init.cc:377] Logged in from keytab as kudu/127.7.132.129@KRBTEST.COM (short username kudu)
I20260812 02:36:22.423082 10809 webserver.cc:460] Webserver started at http://127.7.132.129:33825/ using document root <none> and password file <none>
I20260812 02:36:22.423877 10809 fs_manager.cc:359] Metadata directory not provided
I20260812 02:36:22.423986 10809 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:36:22.424245 10809 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:36:22.427911 10809 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502156971023-7698-0/minicluster-data/ts-0/data/instance:
uuid: "9f6ca1e5cacb4da2bec04fe8020e011d"
format_stamp: "Formatted at 2026-08-12 02:36:22 on dist-test-slave-mn4r"
server_key: "33ce99e89ac333379a7b67d7aeb591da"
server_key_iv: "b7309bb283eff81a515d8fe4b6b5a316"
server_key_version: "encryptionkey@0"
I20260812 02:36:22.428982 10809 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502156971023-7698-0/minicluster-data/ts-0/wal/instance:
uuid: "9f6ca1e5cacb4da2bec04fe8020e011d"
format_stamp: "Formatted at 2026-08-12 02:36:22 on dist-test-slave-mn4r"
server_key: "33ce99e89ac333379a7b67d7aeb591da"
server_key_iv: "b7309bb283eff81a515d8fe4b6b5a316"
server_key_version: "encryptionkey@0"
I20260812 02:36:22.434854 10809 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260812 02:36:22.438530 10827 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:36:22.439888 10809 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260812 02:36:22.440080 10809 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502156971023-7698-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502156971023-7698-0/minicluster-data/ts-0/wal
uuid: "9f6ca1e5cacb4da2bec04fe8020e011d"
format_stamp: "Formatted at 2026-08-12 02:36:22 on dist-test-slave-mn4r"
server_key: "33ce99e89ac333379a7b67d7aeb591da"
server_key_iv: "b7309bb283eff81a515d8fe4b6b5a316"
server_key_version: "encryptionkey@0"
I20260812 02:36:22.440299 10809 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502156971023-7698-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502156971023-7698-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502156971023-7698-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:36:22.462738 10809 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:36:22.467044 10809 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:36:22.467355 10809 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:36:22.468364 10809 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:36:22.469849 10809 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:36:22.469954 10809 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:36:22.470152 10809 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:36:22.470230 10809 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260812 02:36:22.505303 10809 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.129:44859
I20260812 02:36:22.505358 10963 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.129:44859 every 8 connection(s)
I20260812 02:36:22.506760 10809 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502156971023-7698-0/minicluster-data/ts-0/data/info.pb
I20260812 02:36:22.508661 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 10809
I20260812 02:36:22.508894 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502156971023-7698-0/minicluster-data/ts-0/wal/instance
I20260812 02:36:22.509658 7698 external_mini_cluster.cc:1428] Setting key 19e4b3c2b0e9191db0514dfd849fbbf0
Aug 12 02:36:22 dist-test-slave-mn4r krb5kdc[10685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502182, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.129@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 02:36:22.530725 10966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:22.509683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:41823 (local address 127.7.132.190:36655)
0812 02:36:22.510237 (+ 554us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:22.510244 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:22.511443 (+ 1199us) server_negotiation.cc:407] Connection header received
0812 02:36:22.513427 (+ 1984us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:22.513436 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:22.513530 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:22.513663 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:22.515271 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:22.516449 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:22.518124 (+ 1675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:22.518797 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:22.518898 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:22.523300 (+ 4402us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:22.523310 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:22.523319 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:22.523366 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:22.527168 (+ 3802us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:22.527920 (+ 752us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:22.527928 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:22.527934 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:22.528091 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:22.528526 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:22.528534 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:22.528541 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:22.528945 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:22.529123 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:22.529894 (+ 771us) server_negotiation.cc:299] Negotiation successful
0812 02:36:22.530200 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":414,"thread_start_us":252,"threads_started":1}
I20260812 02:36:22.531517 10967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:22.510347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:36655 (local address 127.7.132.129:41823)
0812 02:36:22.511274 (+ 927us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:22.511337 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:22.513143 (+ 1806us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:22.513840 (+ 697us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:22.513849 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:22.514511 (+ 662us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:36:22.515085 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:22.515104 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:22.516612 (+ 1508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:22.516615 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:22.517933 (+ 1318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:22.517945 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:22.519042 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:22.519052 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:22.519382 (+ 330us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:22.520168 (+ 786us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:36:22.520204 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:36:22.523024 (+ 2820us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:36:22.527308 (+ 4284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:22.527315 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:22.527328 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:22.527772 (+ 444us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:22.528201 (+ 429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:22.528204 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:22.528206 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:22.528390 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:22.529121 (+ 731us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:36:22.529127 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:36:22.529460 (+ 333us) client_negotiation.cc:769] Sending connection context
0812 02:36:22.529696 (+ 236us) client_negotiation.cc:240] Negotiation successful
0812 02:36:22.530164 (+ 468us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":643,"thread_start_us":398,"threads_started":1}
WARNING: no policy specified for kudu/127.7.132.130@KRBTEST.COM; defaulting to no policy
I20260812 02:36:22.533212 10964 heartbeater.cc:347] Connected to a master server at 127.7.132.190:36655
I20260812 02:36:22.533524 10964 heartbeater.cc:464] Registering TS with master...
I20260812 02:36:22.534400 10964 heartbeater.cc:511] Master 127.7.132.190:36655 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.130@KRBTEST.COM" created.
I20260812 02:36:22.536679 10738 ts_manager.cc:194] Registered new tserver with Master: 9f6ca1e5cacb4da2bec04fe8020e011d (127.7.132.129:44859)
I20260812 02:36:22.538650 10738 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.129@KRBTEST.COM'} at 127.7.132.129:41823
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.130.keytab.
Entry for principal kudu/127.7.132.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.130.keytab.
I20260812 02:36:22.562024 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502156971023-7698-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502156971023-7698-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502156971023-7698-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502156971023-7698-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.130:0
--local_ip_for_outbound_sockets=127.7.132.130
--webserver_interface=127.7.132.130
--webserver_port=0
--tserver_master_addrs=127.7.132.190:36655
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.130.keytab
--principal=kudu/127.7.132.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:36441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:36:22.896097 10974 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:36:22.896376 10974 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:36:22.896570 10974 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:36:22.903863 10974 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:36:22.903996 10974 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:36:22.904171 10974 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.130
I20260812 02:36:22.914304 10974 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:36441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502156971023-7698-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502156971023-7698-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.130.keytab
--principal=kudu/127.7.132.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502156971023-7698-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.132.130
--webserver_port=0
--tserver_master_addrs=127.7.132.190:36655
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.130
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502156971023-7698-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:36:22.915697 10974 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:36:22.917052 10974 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:36:22.931334 10980 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:36:22.931258 10981 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:36:22.931308 10984 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:36:22.932233 10974 server_base.cc:1034] running on GCE node
I20260812 02:36:22.932937 10974 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:36:22.933872 10974 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:36:22.935101 10974 hybrid_clock.cc:648] HybridClock initialized: now 1786502182935058 us; error 57 us; skew 500 ppm
I20260812 02:36:22.935428 10974 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:36:22 dist-test-slave-mn4r krb5kdc[10685](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502182, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:36:22.938812 10974 init.cc:377] Logged in from keytab as kudu/127.7.132.130@KRBTEST.COM (short username kudu)
I20260812 02:36:22.940207 10974 webserver.cc:460] Webserver started at http://127.7.132.130:34943/ using document root <none> and password file <none>
I20260812 02:36:22.940891 10974 fs_manager.cc:359] Metadata directory not provided
I20260812 02:36:22.940990 10974 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:36:22.941248 10974 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:36:22.944288 10974 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502156971023-7698-0/minicluster-data/ts-1/data/instance:
uuid: "d0af04ea8ab04f889ac381735c17bd35"
format_stamp: "Formatted at 2026-08-12 02:36:22 on dist-test-slave-mn4r"
server_key: "803f52988695a57ff2ed6c3b94f65e20"
server_key_iv: "83a174d1c935aa4974fc9869a83d1d74"
server_key_version: "encryptionkey@0"
I20260812 02:36:22.945103 10974 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502156971023-7698-0/minicluster-data/ts-1/wal/instance:
uuid: "d0af04ea8ab04f889ac381735c17bd35"
format_stamp: "Formatted at 2026-08-12 02:36:22 on dist-test-slave-mn4r"
server_key: "803f52988695a57ff2ed6c3b94f65e20"
server_key_iv: "83a174d1c935aa4974fc9869a83d1d74"
server_key_version: "encryptionkey@0"
I20260812 02:36:22.950186 10974 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.006s
I20260812 02:36:22.953469 10995 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:36:22.954819 10974 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 02:36:22.954991 10974 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502156971023-7698-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502156971023-7698-0/minicluster-data/ts-1/wal
uuid: "d0af04ea8ab04f889ac381735c17bd35"
format_stamp: "Formatted at 2026-08-12 02:36:22 on dist-test-slave-mn4r"
server_key: "803f52988695a57ff2ed6c3b94f65e20"
server_key_iv: "83a174d1c935aa4974fc9869a83d1d74"
server_key_version: "encryptionkey@0"
I20260812 02:36:22.955199 10974 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502156971023-7698-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502156971023-7698-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502156971023-7698-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:36:22.981180 10974 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:36:22.985872 10974 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:36:22.986254 10974 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:36:22.987257 10974 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:36:22.988653 10974 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:36:22.988759 10974 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:36:22.988916 10974 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:36:22.988988 10974 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:36:23.023319 10974 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.130:44677
I20260812 02:36:23.023370 11136 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.130:44677 every 8 connection(s)
I20260812 02:36:23.024716 10974 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502156971023-7698-0/minicluster-data/ts-1/data/info.pb
I20260812 02:36:23.032306 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 10974
I20260812 02:36:23.032541 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502156971023-7698-0/minicluster-data/ts-1/wal/instance
I20260812 02:36:23.032981 7698 external_mini_cluster.cc:1428] Setting key aa1578b2acbf8f55d8c74611bedc740a
Aug 12 02:36:23 dist-test-slave-mn4r krb5kdc[10685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502182, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.130@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 02:36:23.047853 10966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:23.027441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:56297 (local address 127.7.132.190:36655)
0812 02:36:23.027627 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:23.027631 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:23.028821 (+ 1190us) server_negotiation.cc:407] Connection header received
0812 02:36:23.030787 (+ 1966us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:23.030793 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:23.030888 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:23.031007 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:23.032998 (+ 1991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:23.033875 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:23.036082 (+ 2207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:23.036523 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:23.036635 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:23.040998 (+ 4363us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:23.041008 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:23.041016 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:23.041072 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:23.044403 (+ 3331us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:23.045441 (+ 1038us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:23.045445 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:23.045447 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:23.045577 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:23.046068 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:23.046071 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:23.046073 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:23.046385 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:23.046493 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:23.047451 (+ 958us) server_negotiation.cc:299] Negotiation successful
0812 02:36:23.047618 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 02:36:23.049568 11139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:23.027896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:36655 (local address 127.7.132.130:56297)
0812 02:36:23.028639 (+ 743us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:23.028683 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:23.030506 (+ 1823us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:23.031326 (+ 820us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:23.031335 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:23.032226 (+ 891us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:36:23.032819 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:23.032838 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:23.034042 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:23.034046 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:23.035877 (+ 1831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:23.035896 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:23.036778 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:23.036787 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:23.037146 (+ 359us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:23.037920 (+ 774us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:36:23.037955 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:36:23.040755 (+ 2800us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:36:23.044538 (+ 3783us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:23.044550 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:23.044570 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:23.045315 (+ 745us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:23.045699 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:23.045706 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:23.045713 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:23.045952 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:23.046523 (+ 571us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:36:23.046533 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:36:23.047024 (+ 491us) client_negotiation.cc:769] Sending connection context
0812 02:36:23.047338 (+ 314us) client_negotiation.cc:240] Negotiation successful
0812 02:36:23.047967 (+ 629us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":282,"threads_started":1}
I20260812 02:36:23.051206 11137 heartbeater.cc:347] Connected to a master server at 127.7.132.190:36655
I20260812 02:36:23.051537 11137 heartbeater.cc:464] Registering TS with master...
I20260812 02:36:23.052284 11137 heartbeater.cc:511] Master 127.7.132.190:36655 requested a full tablet report, sending...
I20260812 02:36:23.053783 10738 ts_manager.cc:194] Registered new tserver with Master: d0af04ea8ab04f889ac381735c17bd35 (127.7.132.130:44677)
I20260812 02:36:23.054653 10738 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.130@KRBTEST.COM'} at 127.7.132.130:56297
WARNING: no policy specified for kudu/127.7.132.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.131.keytab.
Entry for principal kudu/127.7.132.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.131.keytab.
I20260812 02:36:23.086010 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502156971023-7698-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502156971023-7698-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502156971023-7698-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502156971023-7698-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.131:0
--local_ip_for_outbound_sockets=127.7.132.131
--webserver_interface=127.7.132.131
--webserver_port=0
--tserver_master_addrs=127.7.132.190:36655
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.131.keytab
--principal=kudu/127.7.132.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:36441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:36:23.441402 11146 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:36:23.441697 11146 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:36:23.442060 11146 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:36:23.450723 11146 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:36:23.450862 11146 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:36:23.451071 11146 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.131
I20260812 02:36:23.462018 11146 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:36441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502156971023-7698-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502156971023-7698-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.131.keytab
--principal=kudu/127.7.132.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502156971023-7698-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.132.131
--webserver_port=0
--tserver_master_addrs=127.7.132.190:36655
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.131
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502156971023-7698-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:36:23.463524 11146 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:36:23.465059 11146 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:36:23.479894 11155 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:36:23.479993 11151 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:36:23.479854 11157 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:36:23.480844 11146 server_base.cc:1034] running on GCE node
I20260812 02:36:23.481492 11146 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:36:23.482558 11146 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:36:23.483789 11146 hybrid_clock.cc:648] HybridClock initialized: now 1786502183483735 us; error 47 us; skew 500 ppm
I20260812 02:36:23.484129 11146 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:36:23 dist-test-slave-mn4r krb5kdc[10685](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502183, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:36:23.487553 11146 init.cc:377] Logged in from keytab as kudu/127.7.132.131@KRBTEST.COM (short username kudu)
I20260812 02:36:23.489159 11146 webserver.cc:460] Webserver started at http://127.7.132.131:33121/ using document root <none> and password file <none>
I20260812 02:36:23.489892 11146 fs_manager.cc:359] Metadata directory not provided
I20260812 02:36:23.490043 11146 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:36:23.490401 11146 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:36:23.493427 11146 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502156971023-7698-0/minicluster-data/ts-2/data/instance:
uuid: "9fb1a8f8b0694a23b180841294ea3fcc"
format_stamp: "Formatted at 2026-08-12 02:36:23 on dist-test-slave-mn4r"
server_key: "3d0ad1187d6c1cbd9d6acdc69cebf9c0"
server_key_iv: "6f647b151f8c1038fae54cbfb70080ce"
server_key_version: "encryptionkey@0"
I20260812 02:36:23.494339 11146 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502156971023-7698-0/minicluster-data/ts-2/wal/instance:
uuid: "9fb1a8f8b0694a23b180841294ea3fcc"
format_stamp: "Formatted at 2026-08-12 02:36:23 on dist-test-slave-mn4r"
server_key: "3d0ad1187d6c1cbd9d6acdc69cebf9c0"
server_key_iv: "6f647b151f8c1038fae54cbfb70080ce"
server_key_version: "encryptionkey@0"
I20260812 02:36:23.500020 11146 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260812 02:36:23.503674 11166 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:36:23.505185 11146 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 02:36:23.505390 11146 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502156971023-7698-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502156971023-7698-0/minicluster-data/ts-2/wal
uuid: "9fb1a8f8b0694a23b180841294ea3fcc"
format_stamp: "Formatted at 2026-08-12 02:36:23 on dist-test-slave-mn4r"
server_key: "3d0ad1187d6c1cbd9d6acdc69cebf9c0"
server_key_iv: "6f647b151f8c1038fae54cbfb70080ce"
server_key_version: "encryptionkey@0"
I20260812 02:36:23.505607 11146 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502156971023-7698-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502156971023-7698-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502156971023-7698-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:36:23.521948 11146 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:36:23.526731 11146 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:36:23.527082 11146 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:36:23.528425 11146 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:36:23.530455 11146 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:36:23.530556 11146 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:36:23.530691 11146 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:36:23.530768 11146 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:36:23.542594 10964 heartbeater.cc:503] Master 127.7.132.190:36655 was elected leader, sending a full tablet report...
I20260812 02:36:23.575980 11146 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.131:44539
I20260812 02:36:23.576035 11298 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.131:44539 every 8 connection(s)
I20260812 02:36:23.577312 11146 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502156971023-7698-0/minicluster-data/ts-2/data/info.pb
I20260812 02:36:23.579057 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 11146
I20260812 02:36:23.579267 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786502156971023-7698-0/minicluster-data/ts-2/wal/instance
I20260812 02:36:23.579784 7698 external_mini_cluster.cc:1428] Setting key 1720fb3257463697b740e7ecb6c1d3ea
Aug 12 02:36:23 dist-test-slave-mn4r krb5kdc[10685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502183, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.131@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 02:36:23.598345 11301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:23.580397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:57693 (local address 127.7.132.190:36655)
0812 02:36:23.580965 (+ 568us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:23.580973 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:23.582152 (+ 1179us) server_negotiation.cc:407] Connection header received
0812 02:36:23.584218 (+ 2066us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:23.584222 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:23.584287 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:23.584377 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:23.585845 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:23.586719 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:23.588235 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:23.588627 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:23.588718 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:23.592267 (+ 3549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:23.592272 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:23.592275 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:23.592308 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:23.595230 (+ 2922us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:23.596011 (+ 781us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:23.596015 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:23.596017 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:23.596144 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:23.596531 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:23.596534 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:23.596536 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:23.596785 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:23.596894 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:23.597680 (+ 786us) server_negotiation.cc:299] Negotiation successful
0812 02:36:23.597955 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":424,"thread_start_us":280,"threads_started":1}
I20260812 02:36:23.599069 11302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:23.581017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:36655 (local address 127.7.132.131:57693)
0812 02:36:23.581946 (+ 929us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:23.582026 (+ 80us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:23.583973 (+ 1947us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:23.584601 (+ 628us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:23.584609 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:23.585136 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:36:23.585701 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:23.585719 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:23.586879 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:23.586883 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:23.588111 (+ 1228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:23.588123 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:23.588811 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:23.588816 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:23.589029 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:23.589651 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:36:23.589673 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:36:23.592069 (+ 2396us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:36:23.595365 (+ 3296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:23.595375 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:23.595393 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:23.595898 (+ 505us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:23.596243 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:23.596247 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:23.596249 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:23.596436 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:23.596889 (+ 453us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:36:23.596895 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:36:23.597202 (+ 307us) client_negotiation.cc:769] Sending connection context
0812 02:36:23.597432 (+ 230us) client_negotiation.cc:240] Negotiation successful
0812 02:36:23.597924 (+ 492us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":605,"thread_start_us":360,"threads_started":1}
I20260812 02:36:23.600582 11299 heartbeater.cc:347] Connected to a master server at 127.7.132.190:36655
I20260812 02:36:23.600888 11299 heartbeater.cc:464] Registering TS with master...
I20260812 02:36:23.601578 11299 heartbeater.cc:511] Master 127.7.132.190:36655 requested a full tablet report, sending...
I20260812 02:36:23.603188 10738 ts_manager.cc:194] Registered new tserver with Master: 9fb1a8f8b0694a23b180841294ea3fcc (127.7.132.131:44539)
I20260812 02:36:23.603966 10738 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.131@KRBTEST.COM'} at 127.7.132.131:57693
I20260812 02:36:23.608129 7698 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260812 02:36:23.645777 11301 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:36:23.643091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33928 (local address 127.7.132.190:36655)
0812 02:36:23.643373 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:23.643380 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:23.643724 (+ 344us) server_negotiation.cc:407] Connection header received
0812 02:36:23.643946 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:23.643951 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:23.644033 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:23.644135 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:23.645474 (+ 1339us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33928: BlockingRecv error: recv got EOF from 127.0.0.1:33928 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
I20260812 02:36:23.649399 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 10809
W20260812 02:36:23.652055 7698 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 10809: Not authorized: Client connection negotiation failed: client connection to 127.7.132.129:44859: server requires authentication, but client does not have Kerberos credentials available
I20260812 02:36:23.652251 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 10809
I20260812 02:36:23.674598 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 10974
W20260812 02:36:23.677244 7698 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 10974: Not authorized: Client connection negotiation failed: client connection to 127.7.132.130:44677: server requires authentication, but client does not have Kerberos credentials available
I20260812 02:36:23.677425 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 10974
I20260812 02:36:23.697168 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 11146
W20260812 02:36:23.700080 7698 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 11146: Not authorized: Client connection negotiation failed: client connection to 127.7.132.131:44539: server requires authentication, but client does not have Kerberos credentials available
I20260812 02:36:23.700259 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 11146
I20260812 02:36:23.721840 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 10702
I20260812 02:36:23.723241 10770 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:36:23.828992 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 10702
2026-08-12T02:36:23Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (3840 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786502156971023-7698-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 02:36:23 dist-test-slave-mn4r krb5kdc[11323](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 02:36:23 dist-test-slave-mn4r krb5kdc[11323](info): set up 2 sockets
Aug 12 02:36:23 dist-test-slave-mn4r krb5kdc[11323](info): commencing operation
krb5kdc: starting...
W20260812 02:36:25.038115 7698 mini_kdc.cc:121] Time spent starting KDC: real 1.160s user 0.000s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786502156971023-7698-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786502156971023-7698-0/krb5kdc/test-user.keytab.
Aug 12 02:36:25 dist-test-slave-mn4r krb5kdc[11323](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502185, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T02:36:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T02:36:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.132.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
I20260812 02:36:25.273542 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786502156971023-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786502156971023-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786502156971023-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:38161
--webserver_interface=127.7.132.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:35401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.132.190:38161
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:36:25.610239 11340 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:36:25.610548 11340 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:36:25.610638 11340 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:36:25.617843 11340 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:36:25.617954 11340 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:36:25.618001 11340 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:36:25.618093 11340 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:36:25.618206 11340 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:36:25.628296 11340 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:35401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786502156971023-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786502156971023-7698-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.132.190:38161
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:38161
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.132.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786502156971023-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:36:25.629614 11340 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:36:25.630939 11340 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:36:25.643268 11345 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:36:25.643213 11349 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:36:25.643293 11346 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:36:25.644155 11340 server_base.cc:1034] running on GCE node
I20260812 02:36:25.644898 11340 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:36:25.646313 11340 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:36:25.647523 11340 hybrid_clock.cc:648] HybridClock initialized: now 1786502185647483 us; error 27 us; skew 500 ppm
I20260812 02:36:25.647890 11340 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:36:25 dist-test-slave-mn4r krb5kdc[11323](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502185, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:36:25.656538 11340 init.cc:377] Logged in from keytab as kudu/127.7.132.190@KRBTEST.COM (short username kudu)
I20260812 02:36:25.658315 11340 webserver.cc:460] Webserver started at http://127.7.132.190:41147/ using document root <none> and password file <none>
I20260812 02:36:25.659085 11340 fs_manager.cc:359] Metadata directory not provided
I20260812 02:36:25.659224 11340 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:36:25.659588 11340 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:36:25.662829 11340 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786502156971023-7698-0/minicluster-data/master-0/data/instance:
uuid: "38c20389a01342eea54e2896819016c1"
format_stamp: "Formatted at 2026-08-12 02:36:25 on dist-test-slave-mn4r"
server_key: "fd64ba1db201fac799821b5f0793c92e"
server_key_iv: "7b4c53f1b5b222ada1bd4eae4e3782e6"
server_key_version: "encryptionkey@0"
I20260812 02:36:25.663771 11340 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786502156971023-7698-0/minicluster-data/master-0/wal/instance:
uuid: "38c20389a01342eea54e2896819016c1"
format_stamp: "Formatted at 2026-08-12 02:36:25 on dist-test-slave-mn4r"
server_key: "fd64ba1db201fac799821b5f0793c92e"
server_key_iv: "7b4c53f1b5b222ada1bd4eae4e3782e6"
server_key_version: "encryptionkey@0"
I20260812 02:36:25.669211 11340 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260812 02:36:25.672814 11358 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:36:25.674233 11340 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260812 02:36:25.674418 11340 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786502156971023-7698-0/minicluster-data/master-0/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786502156971023-7698-0/minicluster-data/master-0/wal
uuid: "38c20389a01342eea54e2896819016c1"
format_stamp: "Formatted at 2026-08-12 02:36:25 on dist-test-slave-mn4r"
server_key: "fd64ba1db201fac799821b5f0793c92e"
server_key_iv: "7b4c53f1b5b222ada1bd4eae4e3782e6"
server_key_version: "encryptionkey@0"
I20260812 02:36:25.674633 11340 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786502156971023-7698-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786502156971023-7698-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786502156971023-7698-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:36:25.692065 11340 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:36:25.696715 11340 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:36:25.697033 11340 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:36:25.716473 11340 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.190:38161
I20260812 02:36:25.716521 11421 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.190:38161 every 8 connection(s)
I20260812 02:36:25.717929 11340 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
I20260812 02:36:25.721844 11423 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:36:25.724524 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 11340
I20260812 02:36:25.724761 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786502156971023-7698-0/minicluster-data/master-0/wal/instance
I20260812 02:36:25.725589 7698 external_mini_cluster.cc:1428] Setting key d74e9037982bd0edb3a831752db9e304
I20260812 02:36:25.734231 11423 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38c20389a01342eea54e2896819016c1: Bootstrap starting.
I20260812 02:36:25.737340 11423 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 38c20389a01342eea54e2896819016c1: Neither blocks nor log segments found. Creating new log.
I20260812 02:36:25.738428 11423 log.cc:824] T 00000000000000000000000000000000 P 38c20389a01342eea54e2896819016c1: Log is configured to *not* fsync() on all Append() calls
Aug 12 02:36:25 dist-test-slave-mn4r krb5kdc[11323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502185, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 02:36:25.741626 11423 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38c20389a01342eea54e2896819016c1: No bootstrap required, opened a new log
I20260812 02:36:25.748826 11428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:25.727743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54794 (local address 127.7.132.190:38161)
0812 02:36:25.728615 (+ 872us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:25.728628 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:25.728675 (+ 47us) server_negotiation.cc:407] Connection header received
0812 02:36:25.730369 (+ 1694us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:25.730392 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:25.730796 (+ 404us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:25.731223 (+ 427us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:25.732782 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:25.734136 (+ 1354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:25.735505 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:25.736219 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:25.736415 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:25.740776 (+ 4361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:25.740788 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:25.740806 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:25.740856 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:25.744389 (+ 3533us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:25.745173 (+ 784us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:25.745188 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:25.745199 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:25.745371 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:25.745793 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:25.745798 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:25.745803 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:25.746463 (+ 660us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:25.746799 (+ 336us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:25.747300 (+ 501us) server_negotiation.cc:299] Negotiation successful
0812 02:36:25.747740 (+ 440us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":650,"thread_start_us":383,"threads_started":1}
I20260812 02:36:25.749594 11423 raft_consensus.cc:348] T 00000000000000000000000000000000 P 38c20389a01342eea54e2896819016c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38c20389a01342eea54e2896819016c1" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 38161 } }
I20260812 02:36:25.749985 11423 raft_consensus.cc:374] T 00000000000000000000000000000000 P 38c20389a01342eea54e2896819016c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:36:25.750126 11423 raft_consensus.cc:729] T 00000000000000000000000000000000 P 38c20389a01342eea54e2896819016c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38c20389a01342eea54e2896819016c1, State: Initialized, Role: FOLLOWER
I20260812 02:36:25.750751 11423 consensus_queue.cc:260] T 00000000000000000000000000000000 P 38c20389a01342eea54e2896819016c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38c20389a01342eea54e2896819016c1" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 38161 } }
I20260812 02:36:25.751010 11423 raft_consensus.cc:388] T 00000000000000000000000000000000 P 38c20389a01342eea54e2896819016c1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 02:36:25.751166 11423 raft_consensus.cc:482] T 00000000000000000000000000000000 P 38c20389a01342eea54e2896819016c1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 02:36:25.751354 11423 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 38c20389a01342eea54e2896819016c1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:36:25.753216 7698 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 02:36:25.754038 11423 raft_consensus.cc:504] T 00000000000000000000000000000000 P 38c20389a01342eea54e2896819016c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38c20389a01342eea54e2896819016c1" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 38161 } }
I20260812 02:36:25.754621 11423 leader_election.cc:302] T 00000000000000000000000000000000 P 38c20389a01342eea54e2896819016c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 38c20389a01342eea54e2896819016c1; no voters:
I20260812 02:36:25.755250 11423 leader_election.cc:288] T 00000000000000000000000000000000 P 38c20389a01342eea54e2896819016c1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 02:36:25.755456 11431 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 38c20389a01342eea54e2896819016c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:36:25.757372 11423 sys_catalog.cc:564] T 00000000000000000000000000000000 P 38c20389a01342eea54e2896819016c1 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:36:25.758407 11431 raft_consensus.cc:686] T 00000000000000000000000000000000 P 38c20389a01342eea54e2896819016c1 [term 1 LEADER]: Becoming Leader. State: Replica: 38c20389a01342eea54e2896819016c1, State: Running, Role: LEADER
I20260812 02:36:25.759018 11431 consensus_queue.cc:237] T 00000000000000000000000000000000 P 38c20389a01342eea54e2896819016c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38c20389a01342eea54e2896819016c1" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 38161 } }
I20260812 02:36:25.762404 11433 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38c20389a01342eea54e2896819016c1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 38c20389a01342eea54e2896819016c1. Latest consensus state: current_term: 1 leader_uuid: "38c20389a01342eea54e2896819016c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38c20389a01342eea54e2896819016c1" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 38161 } } }
I20260812 02:36:25.763396 11433 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38c20389a01342eea54e2896819016c1 [sys.catalog]: This master's current role is: LEADER
I20260812 02:36:25.763170 11432 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38c20389a01342eea54e2896819016c1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "38c20389a01342eea54e2896819016c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38c20389a01342eea54e2896819016c1" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 38161 } } }
I20260812 02:36:25.763568 11432 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38c20389a01342eea54e2896819016c1 [sys.catalog]: This master's current role is: LEADER
I20260812 02:36:25.767807 11444 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:36:25.772658 11444 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 02:36:25.781950 11444 catalog_manager.cc:1347] Generated new cluster ID: 0a174fb9f755487bb3c0c638a9bd918b
I20260812 02:36:25.782063 11444 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 02:36:25.805138 11444 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 02:36:25.807049 11444 catalog_manager.cc:1504] Loading token signing keys...
I20260812 02:36:25.809523 11428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:25.800039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54798 (local address 127.7.132.190:38161)
0812 02:36:25.800300 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:25.800311 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:25.800962 (+ 651us) server_negotiation.cc:407] Connection header received
0812 02:36:25.801199 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:25.801204 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:25.801278 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:25.801404 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:25.802535 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:25.803536 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:25.805472 (+ 1936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:25.806198 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:25.806324 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:25.807655 (+ 1331us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:25.807662 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:25.807667 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:36:25.808635 (+ 968us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:25.808721 (+ 86us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:36:25.808762 (+ 41us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:25.808884 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:25.809056 (+ 172us) server_negotiation.cc:299] Negotiation successful
0812 02:36:25.809185 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":23,"server-negotiator.queue_time_us":88}
I20260812 02:36:25.821568 11444 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 38c20389a01342eea54e2896819016c1: Generated new TSK 0
I20260812 02:36:25.822721 11444 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 02:36:25.832234 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 11340
I20260812 02:36:25.833426 11417 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:36:25.925419 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 11340
2026-08-12T02:36:25Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (2087 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502156971023-7698-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 02:36:26 dist-test-slave-mn4r krb5kdc[11467](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 02:36:26 dist-test-slave-mn4r krb5kdc[11467](info): set up 2 sockets
Aug 12 02:36:26 dist-test-slave-mn4r krb5kdc[11467](info): commencing operation
krb5kdc: starting...
W20260812 02:36:27.122568 7698 mini_kdc.cc:121] Time spent starting KDC: real 1.157s user 0.000s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502156971023-7698-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502156971023-7698-0/krb5kdc/test-user.keytab.
Aug 12 02:36:27 dist-test-slave-mn4r krb5kdc[11467](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502187, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T02:36:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T02:36:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.132.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
I20260812 02:36:27.432977 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502156971023-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502156971023-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502156971023-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:44531
--webserver_interface=127.7.132.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:41619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.132.190:44531
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:36:27.840854 11481 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:36:27.841166 11481 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:36:27.841354 11481 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:36:27.848563 11481 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:36:27.848737 11481 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:36:27.848838 11481 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:36:27.848973 11481 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:36:27.849056 11481 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:36:27.859236 11481 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:41619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502156971023-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502156971023-7698-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.132.190:44531
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:44531
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.132.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502156971023-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:36:27.860661 11481 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:36:27.862149 11481 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:36:27.874910 11488 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:36:27.874873 11492 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:36:27.875470 11487 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:36:27.875732 11481 server_base.cc:1034] running on GCE node
I20260812 02:36:27.876534 11481 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:36:27.878026 11481 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:36:27.879333 11481 hybrid_clock.cc:648] HybridClock initialized: now 1786502187879267 us; error 55 us; skew 500 ppm
I20260812 02:36:27.879757 11481 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:36:27 dist-test-slave-mn4r krb5kdc[11467](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502187, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:36:27.883144 11481 init.cc:377] Logged in from keytab as kudu/127.7.132.190@KRBTEST.COM (short username kudu)
I20260812 02:36:27.884706 11481 webserver.cc:460] Webserver started at http://127.7.132.190:36339/ using document root <none> and password file <none>
I20260812 02:36:27.885367 11481 fs_manager.cc:359] Metadata directory not provided
I20260812 02:36:27.885473 11481 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:36:27.885737 11481 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:36:27.888859 11481 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502156971023-7698-0/minicluster-data/master-0/data/instance:
uuid: "87e21b8047df4ca0acd9b0337dce6597"
format_stamp: "Formatted at 2026-08-12 02:36:27 on dist-test-slave-mn4r"
server_key: "52e5264d0ebe1a0a6ad9fc0a434803d5"
server_key_iv: "6671fdce56d99decb30bb2b1249444e3"
server_key_version: "encryptionkey@0"
I20260812 02:36:27.889624 11481 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502156971023-7698-0/minicluster-data/master-0/wal/instance:
uuid: "87e21b8047df4ca0acd9b0337dce6597"
format_stamp: "Formatted at 2026-08-12 02:36:27 on dist-test-slave-mn4r"
server_key: "52e5264d0ebe1a0a6ad9fc0a434803d5"
server_key_iv: "6671fdce56d99decb30bb2b1249444e3"
server_key_version: "encryptionkey@0"
I20260812 02:36:27.895192 11481 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260812 02:36:27.898905 11499 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:36:27.900391 11481 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 02:36:27.900596 11481 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502156971023-7698-0/minicluster-data/master-0/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502156971023-7698-0/minicluster-data/master-0/wal
uuid: "87e21b8047df4ca0acd9b0337dce6597"
format_stamp: "Formatted at 2026-08-12 02:36:27 on dist-test-slave-mn4r"
server_key: "52e5264d0ebe1a0a6ad9fc0a434803d5"
server_key_iv: "6671fdce56d99decb30bb2b1249444e3"
server_key_version: "encryptionkey@0"
I20260812 02:36:27.900806 11481 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502156971023-7698-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502156971023-7698-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502156971023-7698-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:36:27.927605 11481 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:36:27.932104 11481 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:36:27.932463 11481 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:36:27.951635 11481 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.190:44531
I20260812 02:36:27.951668 11555 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.190:44531 every 8 connection(s)
I20260812 02:36:27.953037 11481 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
I20260812 02:36:27.956866 11556 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:36:27.961045 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 11481
I20260812 02:36:27.961302 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502156971023-7698-0/minicluster-data/master-0/wal/instance
I20260812 02:36:27.962160 7698 external_mini_cluster.cc:1428] Setting key 78cf0c672494302040f3d620696229ff
I20260812 02:36:27.967943 11556 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87e21b8047df4ca0acd9b0337dce6597: Bootstrap starting.
I20260812 02:36:27.971117 11556 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 87e21b8047df4ca0acd9b0337dce6597: Neither blocks nor log segments found. Creating new log.
I20260812 02:36:27.972178 11556 log.cc:824] T 00000000000000000000000000000000 P 87e21b8047df4ca0acd9b0337dce6597: Log is configured to *not* fsync() on all Append() calls
I20260812 02:36:27.975221 11556 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87e21b8047df4ca0acd9b0337dce6597: No bootstrap required, opened a new log
Aug 12 02:36:27 dist-test-slave-mn4r krb5kdc[11467](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502187, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 02:36:27.983594 11556 raft_consensus.cc:348] T 00000000000000000000000000000000 P 87e21b8047df4ca0acd9b0337dce6597 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87e21b8047df4ca0acd9b0337dce6597" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 44531 } }
I20260812 02:36:27.984021 11556 raft_consensus.cc:374] T 00000000000000000000000000000000 P 87e21b8047df4ca0acd9b0337dce6597 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:36:27.984141 11556 raft_consensus.cc:729] T 00000000000000000000000000000000 P 87e21b8047df4ca0acd9b0337dce6597 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87e21b8047df4ca0acd9b0337dce6597, State: Initialized, Role: FOLLOWER
I20260812 02:36:27.984800 11556 consensus_queue.cc:260] T 00000000000000000000000000000000 P 87e21b8047df4ca0acd9b0337dce6597 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87e21b8047df4ca0acd9b0337dce6597" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 44531 } }
I20260812 02:36:27.985095 11556 raft_consensus.cc:388] T 00000000000000000000000000000000 P 87e21b8047df4ca0acd9b0337dce6597 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 02:36:27.985275 11556 raft_consensus.cc:482] T 00000000000000000000000000000000 P 87e21b8047df4ca0acd9b0337dce6597 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 02:36:27.985471 11556 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 87e21b8047df4ca0acd9b0337dce6597 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:36:27.987475 11559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:27.964365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50826 (local address 127.7.132.190:44531)
0812 02:36:27.965387 (+ 1022us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:27.965407 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:27.965454 (+ 47us) server_negotiation.cc:407] Connection header received
0812 02:36:27.967338 (+ 1884us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:27.967363 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:27.967776 (+ 413us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:27.968201 (+ 425us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:27.969791 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:27.971339 (+ 1548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:27.972790 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:27.973585 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:27.973814 (+ 229us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:27.978452 (+ 4638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:27.978465 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:27.978481 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:27.978550 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:27.982347 (+ 3797us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:27.983177 (+ 830us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:27.983187 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:27.983194 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:27.983392 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:27.983852 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:27.983858 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:27.983862 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:27.984742 (+ 880us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:27.985167 (+ 425us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:27.985684 (+ 517us) server_negotiation.cc:299] Negotiation successful
0812 02:36:27.986278 (+ 594us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":746,"thread_start_us":507,"threads_started":1}
I20260812 02:36:27.988205 11556 raft_consensus.cc:504] T 00000000000000000000000000000000 P 87e21b8047df4ca0acd9b0337dce6597 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87e21b8047df4ca0acd9b0337dce6597" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 44531 } }
I20260812 02:36:27.988785 11556 leader_election.cc:302] T 00000000000000000000000000000000 P 87e21b8047df4ca0acd9b0337dce6597 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 87e21b8047df4ca0acd9b0337dce6597; no voters:
I20260812 02:36:27.989485 11556 leader_election.cc:288] T 00000000000000000000000000000000 P 87e21b8047df4ca0acd9b0337dce6597 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 02:36:27.989683 11561 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 87e21b8047df4ca0acd9b0337dce6597 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:36:27.991392 11561 raft_consensus.cc:686] T 00000000000000000000000000000000 P 87e21b8047df4ca0acd9b0337dce6597 [term 1 LEADER]: Becoming Leader. State: Replica: 87e21b8047df4ca0acd9b0337dce6597, State: Running, Role: LEADER
I20260812 02:36:27.992034 11561 consensus_queue.cc:237] T 00000000000000000000000000000000 P 87e21b8047df4ca0acd9b0337dce6597 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87e21b8047df4ca0acd9b0337dce6597" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 44531 } }
I20260812 02:36:27.992204 11556 sys_catalog.cc:564] T 00000000000000000000000000000000 P 87e21b8047df4ca0acd9b0337dce6597 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:36:27.995232 11562 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87e21b8047df4ca0acd9b0337dce6597 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "87e21b8047df4ca0acd9b0337dce6597" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87e21b8047df4ca0acd9b0337dce6597" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 44531 } } }
I20260812 02:36:27.995594 11562 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87e21b8047df4ca0acd9b0337dce6597 [sys.catalog]: This master's current role is: LEADER
I20260812 02:36:27.996150 11563 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87e21b8047df4ca0acd9b0337dce6597 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 87e21b8047df4ca0acd9b0337dce6597. Latest consensus state: current_term: 1 leader_uuid: "87e21b8047df4ca0acd9b0337dce6597" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87e21b8047df4ca0acd9b0337dce6597" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 44531 } } }
I20260812 02:36:27.996543 11563 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87e21b8047df4ca0acd9b0337dce6597 [sys.catalog]: This master's current role is: LEADER
I20260812 02:36:27.996613 11571 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:36:28.002876 11571 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 02:36:28.013610 11571 catalog_manager.cc:1347] Generated new cluster ID: db98608f6833443caedf69dbdd000f31
I20260812 02:36:28.013751 11571 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 02:36:28.033680 11571 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 02:36:28.035081 11571 catalog_manager.cc:1504] Loading token signing keys...
I20260812 02:36:28.043916 11571 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 87e21b8047df4ca0acd9b0337dce6597: Generated new TSK 0
I20260812 02:36:28.044804 11571 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.132.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.129.keytab.
Entry for principal kudu/127.7.132.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.129.keytab.
I20260812 02:36:28.106171 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502156971023-7698-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502156971023-7698-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502156971023-7698-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502156971023-7698-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.129:0
--local_ip_for_outbound_sockets=127.7.132.129
--webserver_interface=127.7.132.129
--webserver_port=0
--tserver_master_addrs=127.7.132.190:44531
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.129.keytab
--principal=kudu/127.7.132.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:41619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:36:28.456655 11588 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:36:28.457032 11588 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:36:28.457196 11588 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:36:28.464637 11588 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:36:28.464771 11588 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:36:28.464962 11588 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.129
I20260812 02:36:28.474713 11588 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:41619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502156971023-7698-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502156971023-7698-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.129.keytab
--principal=kudu/127.7.132.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502156971023-7698-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.132.129
--webserver_port=0
--tserver_master_addrs=127.7.132.190:44531
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.129
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502156971023-7698-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:36:28.476150 11588 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:36:28.477699 11588 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:36:28.493105 11598 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:36:28.493014 11596 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:36:28.492976 11594 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:36:28.495700 11588 server_base.cc:1034] running on GCE node
I20260812 02:36:28.496380 11588 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:36:28.497464 11588 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:36:28.498706 11588 hybrid_clock.cc:648] HybridClock initialized: now 1786502188498624 us; error 66 us; skew 500 ppm
I20260812 02:36:28.499148 11588 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:36:28 dist-test-slave-mn4r krb5kdc[11467](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502188, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:36:28.503021 11588 init.cc:377] Logged in from keytab as kudu/127.7.132.129@KRBTEST.COM (short username kudu)
I20260812 02:36:28.504777 11588 webserver.cc:460] Webserver started at http://127.7.132.129:42461/ using document root <none> and password file <none>
I20260812 02:36:28.505636 11588 fs_manager.cc:359] Metadata directory not provided
I20260812 02:36:28.505748 11588 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:36:28.506012 11588 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:36:28.509251 11588 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502156971023-7698-0/minicluster-data/ts-0/data/instance:
uuid: "d521df74814447c8a9d97297c1ad2b3c"
format_stamp: "Formatted at 2026-08-12 02:36:28 on dist-test-slave-mn4r"
server_key: "44842f21f7e9925c87cf7eca720a5a47"
server_key_iv: "857a83bd6f8c233e9ae1647edfa5e5c3"
server_key_version: "encryptionkey@0"
I20260812 02:36:28.510141 11588 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502156971023-7698-0/minicluster-data/ts-0/wal/instance:
uuid: "d521df74814447c8a9d97297c1ad2b3c"
format_stamp: "Formatted at 2026-08-12 02:36:28 on dist-test-slave-mn4r"
server_key: "44842f21f7e9925c87cf7eca720a5a47"
server_key_iv: "857a83bd6f8c233e9ae1647edfa5e5c3"
server_key_version: "encryptionkey@0"
I20260812 02:36:28.515671 11588 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260812 02:36:28.519502 11606 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:36:28.520893 11588 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260812 02:36:28.521081 11588 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502156971023-7698-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502156971023-7698-0/minicluster-data/ts-0/wal
uuid: "d521df74814447c8a9d97297c1ad2b3c"
format_stamp: "Formatted at 2026-08-12 02:36:28 on dist-test-slave-mn4r"
server_key: "44842f21f7e9925c87cf7eca720a5a47"
server_key_iv: "857a83bd6f8c233e9ae1647edfa5e5c3"
server_key_version: "encryptionkey@0"
I20260812 02:36:28.521293 11588 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502156971023-7698-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502156971023-7698-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502156971023-7698-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:36:28.544273 11588 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:36:28.548844 11588 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:36:28.549165 11588 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:36:28.550208 11588 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:36:28.551563 11588 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:36:28.551677 11588 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:36:28.551832 11588 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:36:28.551908 11588 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:36:28.586666 11588 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.129:36747
I20260812 02:36:28.586711 11736 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.129:36747 every 8 connection(s)
I20260812 02:36:28.588033 11588 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502156971023-7698-0/minicluster-data/ts-0/data/info.pb
I20260812 02:36:28.590791 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 11588
I20260812 02:36:28.591030 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502156971023-7698-0/minicluster-data/ts-0/wal/instance
I20260812 02:36:28.591898 7698 external_mini_cluster.cc:1428] Setting key 6eae050bddc3b876ade554e05820706d
Aug 12 02:36:28 dist-test-slave-mn4r krb5kdc[11467](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502188, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.129@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 02:36:28.612067 11739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:28.590851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:48783 (local address 127.7.132.190:44531)
0812 02:36:28.591360 (+ 509us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:28.591365 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:28.593000 (+ 1635us) server_negotiation.cc:407] Connection header received
0812 02:36:28.594920 (+ 1920us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:28.594936 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:28.595039 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:28.595166 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:28.596656 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:28.597833 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:28.599698 (+ 1865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:28.600431 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:28.600583 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:28.604377 (+ 3794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:28.604387 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:28.604395 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:28.604444 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:28.608414 (+ 3970us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:28.609242 (+ 828us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:28.609251 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:28.609258 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:28.609439 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:28.609913 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:28.609922 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:28.609928 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:28.610368 (+ 440us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:28.610539 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:28.611312 (+ 773us) server_negotiation.cc:299] Negotiation successful
0812 02:36:28.611586 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":397,"thread_start_us":267,"threads_started":1}
I20260812 02:36:28.612875 11740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:28.591594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:44531 (local address 127.7.132.129:48783)
0812 02:36:28.592487 (+ 893us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:28.592535 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:28.594633 (+ 2098us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:28.595375 (+ 742us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:28.595384 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:28.595897 (+ 513us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:36:28.596472 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:28.596489 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:28.597997 (+ 1508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:28.598000 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:28.599517 (+ 1517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:28.599531 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:28.600682 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:28.600688 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:28.600945 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:28.601670 (+ 725us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:36:28.601695 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:36:28.604058 (+ 2363us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:36:28.608581 (+ 4523us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:28.608588 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:28.608602 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:28.609087 (+ 485us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:28.609568 (+ 481us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:28.609572 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:28.609574 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:28.609783 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:28.610521 (+ 738us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:36:28.610527 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:36:28.610872 (+ 345us) client_negotiation.cc:769] Sending connection context
0812 02:36:28.611128 (+ 256us) client_negotiation.cc:240] Negotiation successful
0812 02:36:28.611560 (+ 432us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":634,"thread_start_us":331,"threads_started":1}
I20260812 02:36:28.614660 11737 heartbeater.cc:347] Connected to a master server at 127.7.132.190:44531
I20260812 02:36:28.615032 11737 heartbeater.cc:464] Registering TS with master...
I20260812 02:36:28.615797 11737 heartbeater.cc:511] Master 127.7.132.190:44531 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.7.132.130@KRBTEST.COM; defaulting to no policy
I20260812 02:36:28.618045 11516 ts_manager.cc:194] Registered new tserver with Master: d521df74814447c8a9d97297c1ad2b3c (127.7.132.129:36747)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.130@KRBTEST.COM" created.
I20260812 02:36:28.619974 11516 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.129@KRBTEST.COM'} at 127.7.132.129:48783
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.130.keytab.
Entry for principal kudu/127.7.132.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.130.keytab.
I20260812 02:36:28.647898 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502156971023-7698-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502156971023-7698-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502156971023-7698-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502156971023-7698-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.130:0
--local_ip_for_outbound_sockets=127.7.132.130
--webserver_interface=127.7.132.130
--webserver_port=0
--tserver_master_addrs=127.7.132.190:44531
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.130.keytab
--principal=kudu/127.7.132.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:41619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:36:28.982939 11744 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:36:28.983261 11744 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:36:28.983429 11744 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:36:28.991329 11744 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:36:28.991475 11744 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:36:28.991673 11744 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.130
I20260812 02:36:29.001657 11744 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:41619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502156971023-7698-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502156971023-7698-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.130.keytab
--principal=kudu/127.7.132.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502156971023-7698-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.132.130
--webserver_port=0
--tserver_master_addrs=127.7.132.190:44531
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.130
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502156971023-7698-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:36:29.003168 11744 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:36:29.004624 11744 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:36:29.018373 11751 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:36:29.018671 11750 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:36:29.018680 11744 server_base.cc:1034] running on GCE node
W20260812 02:36:29.018344 11754 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:36:29.019634 11744 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:36:29.020546 11744 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:36:29.021751 11744 hybrid_clock.cc:648] HybridClock initialized: now 1786502189021683 us; error 55 us; skew 500 ppm
I20260812 02:36:29.022146 11744 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:36:29 dist-test-slave-mn4r krb5kdc[11467](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502189, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:36:29.025527 11744 init.cc:377] Logged in from keytab as kudu/127.7.132.130@KRBTEST.COM (short username kudu)
I20260812 02:36:29.027014 11744 webserver.cc:460] Webserver started at http://127.7.132.130:44849/ using document root <none> and password file <none>
I20260812 02:36:29.027735 11744 fs_manager.cc:359] Metadata directory not provided
I20260812 02:36:29.027887 11744 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:36:29.028223 11744 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:36:29.031306 11744 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502156971023-7698-0/minicluster-data/ts-1/data/instance:
uuid: "824f4d44406845b28fe5cf0332772a1b"
format_stamp: "Formatted at 2026-08-12 02:36:29 on dist-test-slave-mn4r"
server_key: "d85e867f4dd40ff132f8552acef42445"
server_key_iv: "11537ce71d2dc6463e529739fc4ad0e7"
server_key_version: "encryptionkey@0"
I20260812 02:36:29.032106 11744 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502156971023-7698-0/minicluster-data/ts-1/wal/instance:
uuid: "824f4d44406845b28fe5cf0332772a1b"
format_stamp: "Formatted at 2026-08-12 02:36:29 on dist-test-slave-mn4r"
server_key: "d85e867f4dd40ff132f8552acef42445"
server_key_iv: "11537ce71d2dc6463e529739fc4ad0e7"
server_key_version: "encryptionkey@0"
I20260812 02:36:29.037537 11744 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260812 02:36:29.041262 11762 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:36:29.042733 11744 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 02:36:29.042918 11744 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502156971023-7698-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502156971023-7698-0/minicluster-data/ts-1/wal
uuid: "824f4d44406845b28fe5cf0332772a1b"
format_stamp: "Formatted at 2026-08-12 02:36:29 on dist-test-slave-mn4r"
server_key: "d85e867f4dd40ff132f8552acef42445"
server_key_iv: "11537ce71d2dc6463e529739fc4ad0e7"
server_key_version: "encryptionkey@0"
I20260812 02:36:29.043123 11744 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502156971023-7698-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502156971023-7698-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502156971023-7698-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:36:29.057355 11744 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:36:29.061352 11744 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:36:29.061666 11744 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:36:29.062657 11744 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:36:29.064026 11744 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:36:29.064129 11744 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:36:29.064280 11744 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:36:29.064368 11744 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:36:29.099907 11744 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.130:40065
I20260812 02:36:29.099920 11895 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.130:40065 every 8 connection(s)
I20260812 02:36:29.101423 11744 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502156971023-7698-0/minicluster-data/ts-1/data/info.pb
I20260812 02:36:29.110332 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 11744
I20260812 02:36:29.110523 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502156971023-7698-0/minicluster-data/ts-1/wal/instance
I20260812 02:36:29.111029 7698 external_mini_cluster.cc:1428] Setting key f274ac5567fe25db18d27f00e4de0e6f
Aug 12 02:36:29 dist-test-slave-mn4r krb5kdc[11467](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502189, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.130@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 02:36:29.123660 11739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:29.104195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:59515 (local address 127.7.132.190:44531)
0812 02:36:29.104415 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:29.104420 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:29.105794 (+ 1374us) server_negotiation.cc:407] Connection header received
0812 02:36:29.107705 (+ 1911us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:29.107712 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:29.107813 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:29.107930 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:29.109505 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:29.110566 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:29.112297 (+ 1731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:29.112817 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:29.112938 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:29.117309 (+ 4371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:29.117314 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:29.117318 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:29.117351 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:29.119998 (+ 2647us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:29.120984 (+ 986us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:29.120991 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:29.120997 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:29.121176 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:29.121600 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:29.121607 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:29.121610 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:29.122002 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:29.122212 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:29.122889 (+ 677us) server_negotiation.cc:299] Negotiation successful
0812 02:36:29.123171 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":85}
I20260812 02:36:29.124302 11898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:29.104695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:44531 (local address 127.7.132.130:59515)
0812 02:36:29.105576 (+ 881us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:29.105625 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:29.107400 (+ 1775us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:29.108180 (+ 780us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:29.108188 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:29.108742 (+ 554us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:36:29.109328 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:29.109344 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:29.110731 (+ 1387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:29.110736 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:29.112144 (+ 1408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:29.112159 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:29.113063 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:29.113072 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:29.113343 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:29.114275 (+ 932us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:36:29.114299 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:36:29.117057 (+ 2758us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:36:29.120154 (+ 3097us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:29.120166 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:29.120188 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:29.120844 (+ 656us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:29.121298 (+ 454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:29.121301 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:29.121303 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:29.121488 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:29.122182 (+ 694us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:36:29.122188 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:36:29.122512 (+ 324us) client_negotiation.cc:769] Sending connection context
0812 02:36:29.122747 (+ 235us) client_negotiation.cc:240] Negotiation successful
0812 02:36:29.123159 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":557,"thread_start_us":351,"threads_started":1}
I20260812 02:36:29.125939 11896 heartbeater.cc:347] Connected to a master server at 127.7.132.190:44531
I20260812 02:36:29.126304 11896 heartbeater.cc:464] Registering TS with master...
I20260812 02:36:29.127283 11896 heartbeater.cc:511] Master 127.7.132.190:44531 requested a full tablet report, sending...
I20260812 02:36:29.128980 11516 ts_manager.cc:194] Registered new tserver with Master: 824f4d44406845b28fe5cf0332772a1b (127.7.132.130:40065)
I20260812 02:36:29.130340 11516 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.130@KRBTEST.COM'} at 127.7.132.130:59515
WARNING: no policy specified for kudu/127.7.132.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.131.keytab.
Entry for principal kudu/127.7.132.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.131.keytab.
I20260812 02:36:29.165423 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502156971023-7698-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502156971023-7698-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502156971023-7698-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502156971023-7698-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.131:0
--local_ip_for_outbound_sockets=127.7.132.131
--webserver_interface=127.7.132.131
--webserver_port=0
--tserver_master_addrs=127.7.132.190:44531
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.131.keytab
--principal=kudu/127.7.132.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:41619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:36:29.508307 11902 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:36:29.508625 11902 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:36:29.508826 11902 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:36:29.516322 11902 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:36:29.516433 11902 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:36:29.516640 11902 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.131
I20260812 02:36:29.526942 11902 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:41619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502156971023-7698-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502156971023-7698-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.131.keytab
--principal=kudu/127.7.132.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502156971023-7698-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.132.131
--webserver_port=0
--tserver_master_addrs=127.7.132.190:44531
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.131
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502156971023-7698-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:36:29.528254 11902 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:36:29.529656 11902 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:36:29.542933 11907 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:36:29.542932 11908 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:36:29.543112 11912 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:36:29.544207 11902 server_base.cc:1034] running on GCE node
I20260812 02:36:29.544836 11902 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:36:29.545899 11902 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:36:29.547101 11902 hybrid_clock.cc:648] HybridClock initialized: now 1786502189547064 us; error 65 us; skew 500 ppm
I20260812 02:36:29.547457 11902 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:36:29 dist-test-slave-mn4r krb5kdc[11467](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502189, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:36:29.550907 11902 init.cc:377] Logged in from keytab as kudu/127.7.132.131@KRBTEST.COM (short username kudu)
I20260812 02:36:29.552317 11902 webserver.cc:460] Webserver started at http://127.7.132.131:40157/ using document root <none> and password file <none>
I20260812 02:36:29.553045 11902 fs_manager.cc:359] Metadata directory not provided
I20260812 02:36:29.553153 11902 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:36:29.553411 11902 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:36:29.556407 11902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502156971023-7698-0/minicluster-data/ts-2/data/instance:
uuid: "8c887e778dbc435bae7fd1cf16c64aca"
format_stamp: "Formatted at 2026-08-12 02:36:29 on dist-test-slave-mn4r"
server_key: "1efec4d21e93836ddb89ff6f21e20e15"
server_key_iv: "cc6d1bf487ff6990967974a6cbe3e36e"
server_key_version: "encryptionkey@0"
I20260812 02:36:29.557240 11902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502156971023-7698-0/minicluster-data/ts-2/wal/instance:
uuid: "8c887e778dbc435bae7fd1cf16c64aca"
format_stamp: "Formatted at 2026-08-12 02:36:29 on dist-test-slave-mn4r"
server_key: "1efec4d21e93836ddb89ff6f21e20e15"
server_key_iv: "cc6d1bf487ff6990967974a6cbe3e36e"
server_key_version: "encryptionkey@0"
I20260812 02:36:29.562522 11902 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260812 02:36:29.565891 11919 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:36:29.567426 11902 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 02:36:29.567560 11902 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502156971023-7698-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502156971023-7698-0/minicluster-data/ts-2/wal
uuid: "8c887e778dbc435bae7fd1cf16c64aca"
format_stamp: "Formatted at 2026-08-12 02:36:29 on dist-test-slave-mn4r"
server_key: "1efec4d21e93836ddb89ff6f21e20e15"
server_key_iv: "cc6d1bf487ff6990967974a6cbe3e36e"
server_key_version: "encryptionkey@0"
I20260812 02:36:29.567725 11902 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502156971023-7698-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502156971023-7698-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502156971023-7698-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:36:29.590065 11902 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:36:29.594352 11902 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:36:29.594656 11902 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:36:29.595664 11902 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:36:29.597014 11902 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:36:29.597127 11902 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:36:29.597276 11902 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:36:29.597350 11902 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:36:29.624078 11737 heartbeater.cc:503] Master 127.7.132.190:44531 was elected leader, sending a full tablet report...
I20260812 02:36:29.631897 11902 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.131:45257
I20260812 02:36:29.631943 12053 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.131:45257 every 8 connection(s)
I20260812 02:36:29.633289 11902 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502156971023-7698-0/minicluster-data/ts-2/data/info.pb
I20260812 02:36:29.638672 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 11902
I20260812 02:36:29.638860 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786502156971023-7698-0/minicluster-data/ts-2/wal/instance
I20260812 02:36:29.639431 7698 external_mini_cluster.cc:1428] Setting key 34d4eef834b9a947f1a3d5450bc8243f
Aug 12 02:36:29 dist-test-slave-mn4r krb5kdc[11467](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502189, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.131@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 02:36:29.654410 12057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:29.635971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:43899 (local address 127.7.132.190:44531)
0812 02:36:29.636428 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:29.636433 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:29.637363 (+ 930us) server_negotiation.cc:407] Connection header received
0812 02:36:29.639892 (+ 2529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:29.639897 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:29.639996 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:29.640093 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:29.642204 (+ 2111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:29.643135 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:29.644642 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:29.645029 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:29.645123 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:29.648743 (+ 3620us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:29.648748 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:29.648751 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:29.648784 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:29.651256 (+ 2472us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:29.651920 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:29.651924 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:29.651926 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:29.652077 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:29.652445 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:29.652448 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:29.652450 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:29.652728 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:29.652846 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:29.653665 (+ 819us) server_negotiation.cc:299] Negotiation successful
0812 02:36:29.653909 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":247,"threads_started":1}
I20260812 02:36:29.655433 12060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:29.636467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:44531 (local address 127.7.132.131:43899)
0812 02:36:29.637207 (+ 740us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:29.637247 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:29.639529 (+ 2282us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:29.640397 (+ 868us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:29.640406 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:29.641182 (+ 776us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:36:29.641981 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:29.642008 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:29.643275 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:29.643279 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:29.644516 (+ 1237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:29.644528 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:29.645229 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:29.645234 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:29.645425 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:29.646197 (+ 772us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:36:29.646228 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:36:29.648548 (+ 2320us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:36:29.651382 (+ 2834us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:29.651388 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:29.651401 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:29.651813 (+ 412us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:29.652170 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:29.652173 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:29.652175 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:29.652347 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:29.652841 (+ 494us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:36:29.652846 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:36:29.653175 (+ 329us) client_negotiation.cc:769] Sending connection context
0812 02:36:29.653460 (+ 285us) client_negotiation.cc:240] Negotiation successful
0812 02:36:29.653949 (+ 489us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":503,"thread_start_us":300,"threads_started":1}
I20260812 02:36:29.656810 12054 heartbeater.cc:347] Connected to a master server at 127.7.132.190:44531
I20260812 02:36:29.657155 12054 heartbeater.cc:464] Registering TS with master...
I20260812 02:36:29.657924 12054 heartbeater.cc:511] Master 127.7.132.190:44531 requested a full tablet report, sending...
I20260812 02:36:29.659416 11516 ts_manager.cc:194] Registered new tserver with Master: 8c887e778dbc435bae7fd1cf16c64aca (127.7.132.131:45257)
I20260812 02:36:29.660178 11516 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.131@KRBTEST.COM'} at 127.7.132.131:43899
I20260812 02:36:29.668236 7698 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 02:36:29 dist-test-slave-mn4r krb5kdc[11467](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502189, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 12 02:36:29 dist-test-slave-mn4r krb5kdc[11467](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502189, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 02:36:29.723306 12057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:29.709170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50840 (local address 127.7.132.190:44531)
0812 02:36:29.709356 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:29.709361 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:29.709816 (+ 455us) server_negotiation.cc:407] Connection header received
0812 02:36:29.709974 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:29.709979 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:29.710067 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:29.710229 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:29.711634 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:29.712392 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:29.713591 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:29.714030 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:29.714215 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:29.717964 (+ 3749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:29.717970 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:29.717973 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:29.718006 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:29.720500 (+ 2494us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:29.721172 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:29.721179 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:29.721182 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:29.721316 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:29.721710 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:29.721715 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:29.721719 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:29.722128 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:29.722397 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:29.722728 (+ 331us) server_negotiation.cc:299] Negotiation successful
0812 02:36:29.722983 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
W20260812 02:36:29.723949 11516 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:50840
I20260812 02:36:29.726392 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 11588
Aug 12 02:36:29 dist-test-slave-mn4r krb5kdc[11467](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502189, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.7.132.129@KRBTEST.COM
I20260812 02:36:29.742481 12071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:29.727531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47080 (local address 127.7.132.129:36747)
0812 02:36:29.728166 (+ 635us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:29.728173 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:29.728198 (+ 25us) server_negotiation.cc:407] Connection header received
0812 02:36:29.728260 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:29.728266 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:29.728469 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:29.728667 (+ 198us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:29.729847 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:29.730924 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:29.732125 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:29.732700 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:29.732841 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:29.737068 (+ 4227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:29.737079 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:29.737093 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:29.737129 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:29.739888 (+ 2759us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:29.740581 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:29.740587 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:29.740590 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:29.740726 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:29.741073 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:29.741076 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:29.741078 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:29.741406 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:29.741638 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:29.741918 (+ 280us) server_negotiation.cc:299] Negotiation successful
0812 02:36:29.742195 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":497,"thread_start_us":325,"threads_started":1}
W20260812 02:36:29.743799 11731 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:47080
W20260812 02:36:29.744911 7698 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 11588: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 02:36:29.745136 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 11588
I20260812 02:36:29.765771 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 11744
Aug 12 02:36:29 dist-test-slave-mn4r krb5kdc[11467](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502189, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.7.132.130@KRBTEST.COM
I20260812 02:36:29.782451 12072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:29.766804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58864 (local address 127.7.132.130:40065)
0812 02:36:29.767425 (+ 621us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:29.767433 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:29.767462 (+ 29us) server_negotiation.cc:407] Connection header received
0812 02:36:29.767527 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:29.767533 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:29.767754 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:29.767893 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:29.769107 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:29.770220 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:29.771603 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:29.772195 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:29.772292 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:29.776436 (+ 4144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:29.776450 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:29.776469 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:29.776506 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:29.779400 (+ 2894us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:29.780087 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:29.780095 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:29.780098 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:29.780228 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:29.780626 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:29.780632 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:29.780637 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:29.781022 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:29.781346 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:29.781689 (+ 343us) server_negotiation.cc:299] Negotiation successful
0812 02:36:29.781995 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":464,"thread_start_us":322,"threads_started":1}
W20260812 02:36:29.783738 11890 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:58864
W20260812 02:36:29.784859 7698 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 11744: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 02:36:29.785008 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 11744
I20260812 02:36:29.806308 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 11902
Aug 12 02:36:29 dist-test-slave-mn4r krb5kdc[11467](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502189, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.7.132.131@KRBTEST.COM
I20260812 02:36:29.823930 12073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:29.807433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53050 (local address 127.7.132.131:45257)
0812 02:36:29.808124 (+ 691us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:29.808130 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:29.808159 (+ 29us) server_negotiation.cc:407] Connection header received
0812 02:36:29.808223 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:29.808229 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:29.808432 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:29.808623 (+ 191us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:29.810050 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:29.811208 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:29.812543 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:29.813166 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:29.813309 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:29.817760 (+ 4451us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:29.817775 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:29.817794 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:29.817847 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:29.820965 (+ 3118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:29.821681 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:29.821688 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:29.821691 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:29.821817 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:29.822274 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:29.822280 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:29.822283 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:29.822638 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:29.822976 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:29.823283 (+ 307us) server_negotiation.cc:299] Negotiation successful
0812 02:36:29.823569 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":535,"thread_start_us":346,"threads_started":1}
W20260812 02:36:29.825237 12048 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:53050
W20260812 02:36:29.826345 7698 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 11902: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 02:36:29.826524 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 11902
I20260812 02:36:29.846588 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 11481
I20260812 02:36:29.847823 11551 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:36:29.953438 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 11481
2026-08-12T02:36:29Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (4034 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502156971023-7698-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 02:36:30 dist-test-slave-mn4r krb5kdc[12079](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 02:36:30 dist-test-slave-mn4r krb5kdc[12079](info): set up 2 sockets
Aug 12 02:36:30 dist-test-slave-mn4r krb5kdc[12079](info): commencing operation
krb5kdc: starting...
W20260812 02:36:31.152941 7698 mini_kdc.cc:121] Time spent starting KDC: real 1.153s user 0.001s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502156971023-7698-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502156971023-7698-0/krb5kdc/test-user.keytab.
Aug 12 02:36:31 dist-test-slave-mn4r krb5kdc[12079](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502191, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T02:36:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T02:36:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.132.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
I20260812 02:36:31.389252 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502156971023-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502156971023-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502156971023-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:46125
--webserver_interface=127.7.132.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:41495
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.132.190:46125
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:36:31.727033 12095 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:36:31.727373 12095 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:36:31.727543 12095 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:36:31.734709 12095 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:36:31.734851 12095 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:36:31.734964 12095 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:36:31.735093 12095 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:36:31.735177 12095 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:36:31.744887 12095 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:41495
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502156971023-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502156971023-7698-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.132.190:46125
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:46125
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.132.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502156971023-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:36:31.746250 12095 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:36:31.747603 12095 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:36:31.759708 12102 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:36:31.759794 12103 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:36:31.759814 12105 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:36:31.760442 12095 server_base.cc:1034] running on GCE node
I20260812 02:36:31.761216 12095 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:36:31.762846 12095 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:36:31.764086 12095 hybrid_clock.cc:648] HybridClock initialized: now 1786502191764043 us; error 44 us; skew 500 ppm
I20260812 02:36:31.764433 12095 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:36:31 dist-test-slave-mn4r krb5kdc[12079](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502191, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:36:31.771677 12095 init.cc:377] Logged in from keytab as kudu/127.7.132.190@KRBTEST.COM (short username kudu)
I20260812 02:36:31.773283 12095 webserver.cc:460] Webserver started at http://127.7.132.190:45663/ using document root <none> and password file <none>
I20260812 02:36:31.773998 12095 fs_manager.cc:359] Metadata directory not provided
I20260812 02:36:31.774166 12095 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:36:31.774511 12095 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:36:31.777527 12095 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502156971023-7698-0/minicluster-data/master-0/data/instance:
uuid: "1edae90af5d04da1967e0d5887555ecc"
format_stamp: "Formatted at 2026-08-12 02:36:31 on dist-test-slave-mn4r"
server_key: "ece95df23f3309927c6faf715ca17556"
server_key_iv: "e455a7e403a56c85ce321a1adeb3f3fe"
server_key_version: "encryptionkey@0"
I20260812 02:36:31.778399 12095 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502156971023-7698-0/minicluster-data/master-0/wal/instance:
uuid: "1edae90af5d04da1967e0d5887555ecc"
format_stamp: "Formatted at 2026-08-12 02:36:31 on dist-test-slave-mn4r"
server_key: "ece95df23f3309927c6faf715ca17556"
server_key_iv: "e455a7e403a56c85ce321a1adeb3f3fe"
server_key_version: "encryptionkey@0"
I20260812 02:36:31.783532 12095 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260812 02:36:31.787388 12112 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:36:31.788851 12095 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260812 02:36:31.789063 12095 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502156971023-7698-0/minicluster-data/master-0/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502156971023-7698-0/minicluster-data/master-0/wal
uuid: "1edae90af5d04da1967e0d5887555ecc"
format_stamp: "Formatted at 2026-08-12 02:36:31 on dist-test-slave-mn4r"
server_key: "ece95df23f3309927c6faf715ca17556"
server_key_iv: "e455a7e403a56c85ce321a1adeb3f3fe"
server_key_version: "encryptionkey@0"
I20260812 02:36:31.789274 12095 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502156971023-7698-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502156971023-7698-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502156971023-7698-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:36:31.818861 12095 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:36:31.823063 12095 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:36:31.823362 12095 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:36:31.841845 12095 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.190:46125
I20260812 02:36:31.841914 12173 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.190:46125 every 8 connection(s)
I20260812 02:36:31.843297 12095 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
I20260812 02:36:31.847159 12174 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:36:31.852262 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 12095
I20260812 02:36:31.852543 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502156971023-7698-0/minicluster-data/master-0/wal/instance
I20260812 02:36:31.853209 7698 external_mini_cluster.cc:1428] Setting key c6c377d8151923b85645855b768b5f7c
I20260812 02:36:31.859829 12174 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1edae90af5d04da1967e0d5887555ecc: Bootstrap starting.
I20260812 02:36:31.863722 12174 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1edae90af5d04da1967e0d5887555ecc: Neither blocks nor log segments found. Creating new log.
I20260812 02:36:31.864832 12174 log.cc:824] T 00000000000000000000000000000000 P 1edae90af5d04da1967e0d5887555ecc: Log is configured to *not* fsync() on all Append() calls
I20260812 02:36:31.867985 12174 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1edae90af5d04da1967e0d5887555ecc: No bootstrap required, opened a new log
Aug 12 02:36:31 dist-test-slave-mn4r krb5kdc[12079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502191, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 02:36:31.877708 12174 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1edae90af5d04da1967e0d5887555ecc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1edae90af5d04da1967e0d5887555ecc" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 46125 } }
I20260812 02:36:31.878065 12174 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1edae90af5d04da1967e0d5887555ecc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:36:31.878217 12174 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1edae90af5d04da1967e0d5887555ecc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1edae90af5d04da1967e0d5887555ecc, State: Initialized, Role: FOLLOWER
I20260812 02:36:31.878255 12178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:31.855345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45846 (local address 127.7.132.190:46125)
0812 02:36:31.856296 (+ 951us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:31.856313 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:31.856371 (+ 58us) server_negotiation.cc:407] Connection header received
0812 02:36:31.858892 (+ 2521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:31.858921 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:31.859425 (+ 504us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:31.859858 (+ 433us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:31.861312 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:31.863441 (+ 2129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:31.864743 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:31.865780 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:31.865914 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:31.870527 (+ 4613us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:31.870535 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:31.870549 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:31.870584 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:31.873840 (+ 3256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:31.874790 (+ 950us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:31.874798 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:31.874804 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:31.875004 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:31.875407 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:31.875411 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:31.875413 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:31.875979 (+ 566us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:31.876215 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:31.876678 (+ 463us) server_negotiation.cc:299] Negotiation successful
0812 02:36:31.877059 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":656,"thread_start_us":388,"threads_started":1}
I20260812 02:36:31.878865 12174 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1edae90af5d04da1967e0d5887555ecc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1edae90af5d04da1967e0d5887555ecc" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 46125 } }
I20260812 02:36:31.879139 12174 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1edae90af5d04da1967e0d5887555ecc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 02:36:31.879295 12174 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1edae90af5d04da1967e0d5887555ecc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 02:36:31.879494 12174 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1edae90af5d04da1967e0d5887555ecc [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:36:31.881986 12174 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1edae90af5d04da1967e0d5887555ecc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1edae90af5d04da1967e0d5887555ecc" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 46125 } }
I20260812 02:36:31.882597 12174 leader_election.cc:302] T 00000000000000000000000000000000 P 1edae90af5d04da1967e0d5887555ecc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1edae90af5d04da1967e0d5887555ecc; no voters:
I20260812 02:36:31.883198 12174 leader_election.cc:288] T 00000000000000000000000000000000 P 1edae90af5d04da1967e0d5887555ecc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 02:36:31.883339 12181 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1edae90af5d04da1967e0d5887555ecc [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:36:31.884769 12181 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1edae90af5d04da1967e0d5887555ecc [term 1 LEADER]: Becoming Leader. State: Replica: 1edae90af5d04da1967e0d5887555ecc, State: Running, Role: LEADER
I20260812 02:36:31.885353 12174 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1edae90af5d04da1967e0d5887555ecc [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:36:31.885280 12181 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1edae90af5d04da1967e0d5887555ecc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1edae90af5d04da1967e0d5887555ecc" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 46125 } }
I20260812 02:36:31.888680 12185 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1edae90af5d04da1967e0d5887555ecc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1edae90af5d04da1967e0d5887555ecc. Latest consensus state: current_term: 1 leader_uuid: "1edae90af5d04da1967e0d5887555ecc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1edae90af5d04da1967e0d5887555ecc" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 46125 } } }
I20260812 02:36:31.889148 12185 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1edae90af5d04da1967e0d5887555ecc [sys.catalog]: This master's current role is: LEADER
I20260812 02:36:31.888653 12184 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1edae90af5d04da1967e0d5887555ecc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1edae90af5d04da1967e0d5887555ecc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1edae90af5d04da1967e0d5887555ecc" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 46125 } } }
I20260812 02:36:31.890542 12184 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1edae90af5d04da1967e0d5887555ecc [sys.catalog]: This master's current role is: LEADER
I20260812 02:36:31.892669 12194 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:36:31.897028 12194 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 02:36:31.904834 12194 catalog_manager.cc:1347] Generated new cluster ID: b99642f5e8c44223b249b24d37572bb0
I20260812 02:36:31.904985 12194 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 02:36:31.916507 12194 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 02:36:31.917815 12194 catalog_manager.cc:1504] Loading token signing keys...
I20260812 02:36:31.926651 12194 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1edae90af5d04da1967e0d5887555ecc: Generated new TSK 0
I20260812 02:36:31.927520 12194 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.132.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.129.keytab.
Entry for principal kudu/127.7.132.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.129.keytab.
I20260812 02:36:31.994277 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502156971023-7698-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502156971023-7698-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502156971023-7698-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502156971023-7698-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.129:0
--local_ip_for_outbound_sockets=127.7.132.129
--webserver_interface=127.7.132.129
--webserver_port=0
--tserver_master_addrs=127.7.132.190:46125
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.129.keytab
--principal=kudu/127.7.132.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:41495
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:36:32.336640 12208 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:36:32.336963 12208 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:36:32.337059 12208 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:36:32.344316 12208 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:36:32.344424 12208 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:36:32.344534 12208 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.129
I20260812 02:36:32.354045 12208 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:41495
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502156971023-7698-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502156971023-7698-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.129.keytab
--principal=kudu/127.7.132.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502156971023-7698-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.132.129
--webserver_port=0
--tserver_master_addrs=127.7.132.190:46125
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.129
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502156971023-7698-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:36:32.355551 12208 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:36:32.356961 12208 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:36:32.371061 12215 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:36:32.371096 12217 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:36:32.371156 12214 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:36:32.375893 12208 server_base.cc:1034] running on GCE node
I20260812 02:36:32.376649 12208 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:36:32.377933 12208 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:36:32.379172 12208 hybrid_clock.cc:648] HybridClock initialized: now 1786502192379113 us; error 78 us; skew 500 ppm
I20260812 02:36:32.379509 12208 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:36:32 dist-test-slave-mn4r krb5kdc[12079](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502192, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:36:32.382936 12208 init.cc:377] Logged in from keytab as kudu/127.7.132.129@KRBTEST.COM (short username kudu)
I20260812 02:36:32.384546 12208 webserver.cc:460] Webserver started at http://127.7.132.129:46707/ using document root <none> and password file <none>
I20260812 02:36:32.385442 12208 fs_manager.cc:359] Metadata directory not provided
I20260812 02:36:32.385551 12208 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:36:32.385809 12208 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:36:32.389182 12208 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502156971023-7698-0/minicluster-data/ts-0/data/instance:
uuid: "1b5275a908e24c9a8bee22097735e566"
format_stamp: "Formatted at 2026-08-12 02:36:32 on dist-test-slave-mn4r"
server_key: "21686f42379a9036d4107a790500de80"
server_key_iv: "0131c547bd39898593ee1a76240e4f74"
server_key_version: "encryptionkey@0"
I20260812 02:36:32.390018 12208 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502156971023-7698-0/minicluster-data/ts-0/wal/instance:
uuid: "1b5275a908e24c9a8bee22097735e566"
format_stamp: "Formatted at 2026-08-12 02:36:32 on dist-test-slave-mn4r"
server_key: "21686f42379a9036d4107a790500de80"
server_key_iv: "0131c547bd39898593ee1a76240e4f74"
server_key_version: "encryptionkey@0"
I20260812 02:36:32.395465 12208 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260812 02:36:32.398934 12224 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:36:32.400509 12208 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 02:36:32.400681 12208 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502156971023-7698-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502156971023-7698-0/minicluster-data/ts-0/wal
uuid: "1b5275a908e24c9a8bee22097735e566"
format_stamp: "Formatted at 2026-08-12 02:36:32 on dist-test-slave-mn4r"
server_key: "21686f42379a9036d4107a790500de80"
server_key_iv: "0131c547bd39898593ee1a76240e4f74"
server_key_version: "encryptionkey@0"
I20260812 02:36:32.400885 12208 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502156971023-7698-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502156971023-7698-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502156971023-7698-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:36:32.419540 12208 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:36:32.424101 12208 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:36:32.424430 12208 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:36:32.425441 12208 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:36:32.427017 12208 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:36:32.427098 12208 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:36:32.427275 12208 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:36:32.427367 12208 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:36:32.461211 12208 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.129:39499
I20260812 02:36:32.461247 12362 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.129:39499 every 8 connection(s)
I20260812 02:36:32.462657 12208 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502156971023-7698-0/minicluster-data/ts-0/data/info.pb
I20260812 02:36:32.467929 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 12208
I20260812 02:36:32.468173 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502156971023-7698-0/minicluster-data/ts-0/wal/instance
I20260812 02:36:32.468886 7698 external_mini_cluster.cc:1428] Setting key 0b4245681db0ba1cfe3a50532f2af4aa
Aug 12 02:36:32 dist-test-slave-mn4r krb5kdc[12079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502192, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.129@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 02:36:32.485018 12365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:32.465393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:58619 (local address 127.7.132.190:46125)
0812 02:36:32.465896 (+ 503us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:32.465901 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:32.467047 (+ 1146us) server_negotiation.cc:407] Connection header received
0812 02:36:32.469287 (+ 2240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:32.469295 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:32.469426 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:32.469559 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:32.472487 (+ 2928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:32.473219 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:32.474923 (+ 1704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:32.475342 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:32.475482 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:32.478991 (+ 3509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:32.478997 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:32.479000 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:32.479032 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:32.481703 (+ 2671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:32.482563 (+ 860us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:32.482577 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:32.482579 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:32.482707 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:32.483177 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:32.483180 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:32.483183 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:32.483462 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:32.483632 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:32.484384 (+ 752us) server_negotiation.cc:299] Negotiation successful
0812 02:36:32.484631 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":381,"thread_start_us":268,"threads_started":1}
I20260812 02:36:32.486359 12366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:32.466040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:46125 (local address 127.7.132.129:58619)
0812 02:36:32.466861 (+ 821us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:32.466903 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:32.468898 (+ 1995us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:32.470491 (+ 1593us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:32.470505 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:32.471315 (+ 810us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:36:32.472273 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:32.472320 (+ 47us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:32.473389 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:32.473394 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:32.474777 (+ 1383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:32.474791 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:32.475603 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:32.475621 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:32.475855 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:32.476556 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:36:32.476583 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:36:32.478749 (+ 2166us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:36:32.481855 (+ 3106us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:32.481864 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:32.481894 (+ 30us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:32.482425 (+ 531us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:32.482823 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:32.482828 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:32.482832 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:32.483034 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:32.483603 (+ 569us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:36:32.483619 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:36:32.483978 (+ 359us) client_negotiation.cc:769] Sending connection context
0812 02:36:32.484261 (+ 283us) client_negotiation.cc:240] Negotiation successful
0812 02:36:32.484863 (+ 602us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":582,"thread_start_us":294,"threads_started":1}
I20260812 02:36:32.488108 12363 heartbeater.cc:347] Connected to a master server at 127.7.132.190:46125
I20260812 02:36:32.488514 12363 heartbeater.cc:464] Registering TS with master...
I20260812 02:36:32.489581 12363 heartbeater.cc:511] Master 127.7.132.190:46125 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.7.132.130@KRBTEST.COM; defaulting to no policy
I20260812 02:36:32.492317 12132 ts_manager.cc:194] Registered new tserver with Master: 1b5275a908e24c9a8bee22097735e566 (127.7.132.129:39499)
I20260812 02:36:32.494046 12132 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.129@KRBTEST.COM'} at 127.7.132.129:58619
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.130.keytab.
Entry for principal kudu/127.7.132.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.130.keytab.
I20260812 02:36:32.521806 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502156971023-7698-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502156971023-7698-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502156971023-7698-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502156971023-7698-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.130:0
--local_ip_for_outbound_sockets=127.7.132.130
--webserver_interface=127.7.132.130
--webserver_port=0
--tserver_master_addrs=127.7.132.190:46125
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.130.keytab
--principal=kudu/127.7.132.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:41495
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:36:32.857662 12369 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:36:32.857918 12369 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:36:32.858012 12369 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:36:32.865322 12369 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:36:32.865432 12369 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:36:32.865562 12369 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.130
I20260812 02:36:32.875334 12369 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:41495
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502156971023-7698-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502156971023-7698-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.130.keytab
--principal=kudu/127.7.132.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502156971023-7698-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.132.130
--webserver_port=0
--tserver_master_addrs=127.7.132.190:46125
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.130
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502156971023-7698-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:36:32.876616 12369 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:36:32.877892 12369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:36:32.891566 12374 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:36:32.891607 12375 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:36:32.891654 12380 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:36:32.895677 12369 server_base.cc:1034] running on GCE node
I20260812 02:36:32.896337 12369 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:36:32.897356 12369 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:36:32.898633 12369 hybrid_clock.cc:648] HybridClock initialized: now 1786502192898580 us; error 42 us; skew 500 ppm
I20260812 02:36:32.899022 12369 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:36:32 dist-test-slave-mn4r krb5kdc[12079](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502192, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:36:32.902494 12369 init.cc:377] Logged in from keytab as kudu/127.7.132.130@KRBTEST.COM (short username kudu)
I20260812 02:36:32.904175 12369 webserver.cc:460] Webserver started at http://127.7.132.130:42341/ using document root <none> and password file <none>
I20260812 02:36:32.904886 12369 fs_manager.cc:359] Metadata directory not provided
I20260812 02:36:32.904994 12369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:36:32.905333 12369 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:36:32.908368 12369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502156971023-7698-0/minicluster-data/ts-1/data/instance:
uuid: "b1d6f419e6564a0baf4020c22916bb48"
format_stamp: "Formatted at 2026-08-12 02:36:32 on dist-test-slave-mn4r"
server_key: "2ee6976b5f96b27999e91c2a3161951d"
server_key_iv: "3deb99db76f8eb0e5d06f508a1f55ecf"
server_key_version: "encryptionkey@0"
I20260812 02:36:32.909102 12369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502156971023-7698-0/minicluster-data/ts-1/wal/instance:
uuid: "b1d6f419e6564a0baf4020c22916bb48"
format_stamp: "Formatted at 2026-08-12 02:36:32 on dist-test-slave-mn4r"
server_key: "2ee6976b5f96b27999e91c2a3161951d"
server_key_iv: "3deb99db76f8eb0e5d06f508a1f55ecf"
server_key_version: "encryptionkey@0"
I20260812 02:36:32.914793 12369 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260812 02:36:32.918226 12389 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:36:32.919703 12369 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260812 02:36:32.919864 12369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502156971023-7698-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502156971023-7698-0/minicluster-data/ts-1/wal
uuid: "b1d6f419e6564a0baf4020c22916bb48"
format_stamp: "Formatted at 2026-08-12 02:36:32 on dist-test-slave-mn4r"
server_key: "2ee6976b5f96b27999e91c2a3161951d"
server_key_iv: "3deb99db76f8eb0e5d06f508a1f55ecf"
server_key_version: "encryptionkey@0"
I20260812 02:36:32.920068 12369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502156971023-7698-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502156971023-7698-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502156971023-7698-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:36:32.952230 12369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:36:32.956866 12369 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:36:32.957127 12369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:36:32.958055 12369 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:36:32.959451 12369 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:36:32.959525 12369 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:36:32.959611 12369 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:36:32.959663 12369 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:36:32.994544 12369 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.130:46201
I20260812 02:36:32.994594 12521 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.130:46201 every 8 connection(s)
I20260812 02:36:32.995867 12369 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502156971023-7698-0/minicluster-data/ts-1/data/info.pb
I20260812 02:36:33.004686 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 12369
I20260812 02:36:33.004901 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502156971023-7698-0/minicluster-data/ts-1/wal/instance
I20260812 02:36:33.005373 7698 external_mini_cluster.cc:1428] Setting key 04ccbd4175bc9853b3c336001b4bbf37
Aug 12 02:36:33 dist-test-slave-mn4r krb5kdc[12079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502192, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.130@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 02:36:33.016456 12525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:32.998497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:44951 (local address 127.7.132.190:46125)
0812 02:36:32.999004 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:32.999008 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:32.999952 (+ 944us) server_negotiation.cc:407] Connection header received
0812 02:36:33.001835 (+ 1883us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:33.001840 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:33.001946 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:33.002069 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:33.003616 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:33.004366 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:33.006379 (+ 2013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:33.006864 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:33.007005 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:33.010526 (+ 3521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:33.010532 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:33.010535 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:33.010568 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:33.013282 (+ 2714us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:33.014044 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:33.014049 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:33.014051 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:33.014236 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:33.014690 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:33.014695 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:33.014698 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:33.014974 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:33.015145 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:33.015836 (+ 691us) server_negotiation.cc:299] Negotiation successful
0812 02:36:33.016061 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":401,"thread_start_us":276,"threads_started":1}
I20260812 02:36:33.017494 12526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:32.999079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:46125 (local address 127.7.132.130:44951)
0812 02:36:32.999795 (+ 716us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:32.999834 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:33.001548 (+ 1714us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:33.002346 (+ 798us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:33.002354 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:33.002876 (+ 522us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:36:33.003441 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:33.003459 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:33.004547 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:33.004551 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:33.006232 (+ 1681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:33.006245 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:33.007096 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:33.007105 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:33.007362 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:33.008050 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:36:33.008073 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:36:33.010291 (+ 2218us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:36:33.013445 (+ 3154us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:33.013454 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:33.013471 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:33.013897 (+ 426us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:33.014355 (+ 458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:33.014361 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:33.014365 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:33.014563 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:33.015110 (+ 547us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:36:33.015118 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:36:33.015465 (+ 347us) client_negotiation.cc:769] Sending connection context
0812 02:36:33.015735 (+ 270us) client_negotiation.cc:240] Negotiation successful
0812 02:36:33.016282 (+ 547us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":286,"threads_started":1}
I20260812 02:36:33.019058 12523 heartbeater.cc:347] Connected to a master server at 127.7.132.190:46125
I20260812 02:36:33.019384 12523 heartbeater.cc:464] Registering TS with master...
I20260812 02:36:33.020102 12523 heartbeater.cc:511] Master 127.7.132.190:46125 requested a full tablet report, sending...
I20260812 02:36:33.021550 12132 ts_manager.cc:194] Registered new tserver with Master: b1d6f419e6564a0baf4020c22916bb48 (127.7.132.130:46201)
I20260812 02:36:33.022465 12132 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.130@KRBTEST.COM'} at 127.7.132.130:44951
WARNING: no policy specified for kudu/127.7.132.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.131.keytab.
Entry for principal kudu/127.7.132.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.131.keytab.
I20260812 02:36:33.057606 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502156971023-7698-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502156971023-7698-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502156971023-7698-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502156971023-7698-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.131:0
--local_ip_for_outbound_sockets=127.7.132.131
--webserver_interface=127.7.132.131
--webserver_port=0
--tserver_master_addrs=127.7.132.190:46125
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.131.keytab
--principal=kudu/127.7.132.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:41495
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:36:33.407253 12530 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:36:33.407526 12530 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:36:33.407618 12530 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:36:33.414719 12530 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:36:33.414829 12530 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:36:33.414948 12530 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.131
I20260812 02:36:33.425444 12530 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:41495
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502156971023-7698-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502156971023-7698-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.131.keytab
--principal=kudu/127.7.132.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502156971023-7698-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.132.131
--webserver_port=0
--tserver_master_addrs=127.7.132.190:46125
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.131
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502156971023-7698-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:36:33.426857 12530 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:36:33.428247 12530 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:36:33.442164 12535 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:36:33.442065 12536 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:36:33.442319 12538 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:36:33.443010 12530 server_base.cc:1034] running on GCE node
I20260812 02:36:33.443701 12530 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:36:33.444706 12530 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:36:33.445945 12530 hybrid_clock.cc:648] HybridClock initialized: now 1786502193445894 us; error 41 us; skew 500 ppm
I20260812 02:36:33.446316 12530 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:36:33 dist-test-slave-mn4r krb5kdc[12079](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502193, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:36:33.449573 12530 init.cc:377] Logged in from keytab as kudu/127.7.132.131@KRBTEST.COM (short username kudu)
I20260812 02:36:33.451059 12530 webserver.cc:460] Webserver started at http://127.7.132.131:39385/ using document root <none> and password file <none>
I20260812 02:36:33.451748 12530 fs_manager.cc:359] Metadata directory not provided
I20260812 02:36:33.451887 12530 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:36:33.452221 12530 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:36:33.455205 12530 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502156971023-7698-0/minicluster-data/ts-2/data/instance:
uuid: "25ab3790adf44f22af21a4858f326f6e"
format_stamp: "Formatted at 2026-08-12 02:36:33 on dist-test-slave-mn4r"
server_key: "748208b7225faa680c69177fc14b4554"
server_key_iv: "fe180cea129def85cc324555bec2e9fc"
server_key_version: "encryptionkey@0"
I20260812 02:36:33.455994 12530 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502156971023-7698-0/minicluster-data/ts-2/wal/instance:
uuid: "25ab3790adf44f22af21a4858f326f6e"
format_stamp: "Formatted at 2026-08-12 02:36:33 on dist-test-slave-mn4r"
server_key: "748208b7225faa680c69177fc14b4554"
server_key_iv: "fe180cea129def85cc324555bec2e9fc"
server_key_version: "encryptionkey@0"
I20260812 02:36:33.461292 12530 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260812 02:36:33.465799 12545 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:36:33.467398 12530 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 02:36:33.467530 12530 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502156971023-7698-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502156971023-7698-0/minicluster-data/ts-2/wal
uuid: "25ab3790adf44f22af21a4858f326f6e"
format_stamp: "Formatted at 2026-08-12 02:36:33 on dist-test-slave-mn4r"
server_key: "748208b7225faa680c69177fc14b4554"
server_key_iv: "fe180cea129def85cc324555bec2e9fc"
server_key_version: "encryptionkey@0"
I20260812 02:36:33.467697 12530 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502156971023-7698-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502156971023-7698-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502156971023-7698-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:36:33.492383 12530 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:36:33.496815 12530 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:36:33.497171 12530 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:36:33.497239 12363 heartbeater.cc:503] Master 127.7.132.190:46125 was elected leader, sending a full tablet report...
I20260812 02:36:33.498418 12530 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:36:33.500102 12530 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:36:33.500181 12530 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:36:33.500267 12530 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:36:33.500312 12530 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:36:33.533026 12530 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.131:43237
I20260812 02:36:33.533118 12674 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.131:43237 every 8 connection(s)
I20260812 02:36:33.534374 12530 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502156971023-7698-0/minicluster-data/ts-2/data/info.pb
I20260812 02:36:33.541006 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 12530
I20260812 02:36:33.541201 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786502156971023-7698-0/minicluster-data/ts-2/wal/instance
I20260812 02:36:33.541642 7698 external_mini_cluster.cc:1428] Setting key 5ea8229d0875804226433d55eb616f7e
Aug 12 02:36:33 dist-test-slave-mn4r krb5kdc[12079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502193, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.131@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 02:36:33.555727 12678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:33.537000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:56277 (local address 127.7.132.190:46125)
0812 02:36:33.537502 (+ 502us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:33.537508 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:33.538663 (+ 1155us) server_negotiation.cc:407] Connection header received
0812 02:36:33.540468 (+ 1805us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:33.540472 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:33.540542 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:33.540694 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:33.542539 (+ 1845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:33.543761 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:33.545733 (+ 1972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:33.546280 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:33.546413 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:33.549855 (+ 3442us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:33.549860 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:33.549864 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:33.549902 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:33.552599 (+ 2697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:33.553372 (+ 773us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:33.553376 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:33.553378 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:33.553516 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:33.553971 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:33.553976 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:33.553979 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:33.554325 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:33.554463 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:33.555105 (+ 642us) server_negotiation.cc:299] Negotiation successful
0812 02:36:33.555337 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":358,"thread_start_us":221,"threads_started":1}
I20260812 02:36:33.556770 12679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:33.537683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:46125 (local address 127.7.132.131:56277)
0812 02:36:33.538488 (+ 805us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:33.538535 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:33.540195 (+ 1660us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:33.540882 (+ 687us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:33.540890 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:33.541565 (+ 675us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:36:33.542320 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:33.542355 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:33.543945 (+ 1590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:33.543953 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:33.545572 (+ 1619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:33.545585 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:33.546507 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:33.546512 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:33.546708 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:33.547416 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:36:33.547439 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:36:33.549643 (+ 2204us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:36:33.552743 (+ 3100us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:33.552751 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:33.552763 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:33.553263 (+ 500us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:33.553648 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:33.553652 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:33.553654 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:33.553847 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:33.554447 (+ 600us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:36:33.554453 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:36:33.554754 (+ 301us) client_negotiation.cc:769] Sending connection context
0812 02:36:33.555028 (+ 274us) client_negotiation.cc:240] Negotiation successful
0812 02:36:33.555412 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":570,"thread_start_us":362,"threads_started":1}
I20260812 02:36:33.558669 12676 heartbeater.cc:347] Connected to a master server at 127.7.132.190:46125
I20260812 02:36:33.558995 12676 heartbeater.cc:464] Registering TS with master...
I20260812 02:36:33.559711 12676 heartbeater.cc:511] Master 127.7.132.190:46125 requested a full tablet report, sending...
I20260812 02:36:33.561077 12132 ts_manager.cc:194] Registered new tserver with Master: 25ab3790adf44f22af21a4858f326f6e (127.7.132.131:43237)
I20260812 02:36:33.561880 12132 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.131@KRBTEST.COM'} at 127.7.132.131:56277
I20260812 02:36:33.570013 7698 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 02:36:33 dist-test-slave-mn4r krb5kdc[12079](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502193, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 02:36:33 dist-test-slave-mn4r krb5kdc[12079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502193, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.132.129@KRBTEST.COM
I20260812 02:36:33.626917 12684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:33.610878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51730 (local address 127.7.132.129:39499)
0812 02:36:33.611611 (+ 733us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:33.611619 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:33.611647 (+ 28us) server_negotiation.cc:407] Connection header received
0812 02:36:33.611714 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:33.611721 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:33.611977 (+ 256us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:33.612137 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:33.613614 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:33.614885 (+ 1271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:33.616098 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:33.616727 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:33.616836 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:33.620772 (+ 3936us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:33.620784 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:33.620799 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:33.620833 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:33.623938 (+ 3105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:33.624561 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:33.624568 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:33.624570 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:33.624699 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:33.625290 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:33.625294 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:33.625296 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:33.625653 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:33.625889 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:33.626300 (+ 411us) server_negotiation.cc:299] Negotiation successful
0812 02:36:33.626497 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":579,"thread_start_us":372,"threads_started":1}
Aug 12 02:36:33 dist-test-slave-mn4r krb5kdc[12079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502193, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 02:36:33.646039 12678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:33.631795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45862 (local address 127.7.132.190:46125)
0812 02:36:33.632038 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:33.632043 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:33.632141 (+ 98us) server_negotiation.cc:407] Connection header received
0812 02:36:33.632271 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:33.632274 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:33.632339 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:33.632422 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:33.633739 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:33.634585 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:33.635877 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:33.636394 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:33.636497 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:33.640645 (+ 4148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:33.640650 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:33.640653 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:33.640685 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:33.643405 (+ 2720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:33.644069 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:33.644074 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:33.644076 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:33.644209 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:33.644570 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:33.644573 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:33.644575 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:33.644892 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:33.645087 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:33.645436 (+ 349us) server_negotiation.cc:299] Negotiation successful
0812 02:36:33.645689 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
W20260812 02:36:33.646792 12132 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45862
I20260812 02:36:33.647845 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 12208
I20260812 02:36:33.660607 12684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:33.648695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51742 (local address 127.7.132.129:39499)
0812 02:36:33.648849 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:33.648853 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:33.649203 (+ 350us) server_negotiation.cc:407] Connection header received
0812 02:36:33.649294 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:33.649298 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:33.649406 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:33.649497 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:33.650757 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:33.651561 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:33.652726 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:33.653142 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:33.653254 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:33.655606 (+ 2352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:33.655610 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:33.655613 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:33.655645 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:33.658353 (+ 2708us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:33.659028 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:33.659031 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:33.659033 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:33.659156 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:33.659504 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:33.659507 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:33.659509 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:33.659744 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:33.659918 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:33.660200 (+ 282us) server_negotiation.cc:299] Negotiation successful
0812 02:36:33.660367 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 02:36:33.661197 12358 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:36:33.744576 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 12208
I20260812 02:36:33.766160 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 12369
Aug 12 02:36:33 dist-test-slave-mn4r krb5kdc[12079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502193, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.132.130@KRBTEST.COM
I20260812 02:36:33.783747 12689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:33.767230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50200 (local address 127.7.132.130:46201)
0812 02:36:33.767927 (+ 697us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:33.767935 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:33.767964 (+ 29us) server_negotiation.cc:407] Connection header received
0812 02:36:33.768031 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:33.768038 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:33.768273 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:33.768421 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:33.769749 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:33.771036 (+ 1287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:33.772485 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:33.773047 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:33.773157 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:33.777620 (+ 4463us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:33.777632 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:33.777648 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:33.777683 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:33.780746 (+ 3063us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:33.781435 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:33.781443 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:33.781445 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:33.781572 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:33.781985 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:33.781993 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:33.781998 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:33.782425 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:33.782757 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:33.783092 (+ 335us) server_negotiation.cc:299] Negotiation successful
0812 02:36:33.783373 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":525,"thread_start_us":377,"threads_started":1}
I20260812 02:36:33.785092 12516 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:36:33.869736 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 12369
I20260812 02:36:33.892900 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 12530
Aug 12 02:36:33 dist-test-slave-mn4r krb5kdc[12079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502193, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.132.131@KRBTEST.COM
I20260812 02:36:33.911043 12694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:33.894075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52004 (local address 127.7.132.131:43237)
0812 02:36:33.894979 (+ 904us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:33.894988 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:33.895023 (+ 35us) server_negotiation.cc:407] Connection header received
0812 02:36:33.895092 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:33.895099 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:33.895332 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:33.895484 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:33.897015 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:33.898375 (+ 1360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:33.899868 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:33.900496 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:33.900614 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:33.905163 (+ 4549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:33.905177 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:33.905193 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:33.905237 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:33.908158 (+ 2921us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:33.908839 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:33.908846 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:33.908848 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:33.908974 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:33.909371 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:33.909374 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:33.909376 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:33.909722 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:33.909957 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:33.910412 (+ 455us) server_negotiation.cc:299] Negotiation successful
0812 02:36:33.910632 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":629,"thread_start_us":452,"threads_started":1}
I20260812 02:36:33.912480 12670 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:36:33.996668 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 12530
I20260812 02:36:34.019332 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 12095
I20260812 02:36:34.020527 12169 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:36:34.127605 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 12095
2026-08-12T02:36:34Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (4178 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502156971023-7698-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 02:36:34 dist-test-slave-mn4r krb5kdc[12699](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 02:36:34 dist-test-slave-mn4r krb5kdc[12699](info): set up 2 sockets
Aug 12 02:36:34 dist-test-slave-mn4r krb5kdc[12699](info): commencing operation
krb5kdc: starting...
W20260812 02:36:35.437320 7698 mini_kdc.cc:121] Time spent starting KDC: real 1.259s user 0.003s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502156971023-7698-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502156971023-7698-0/krb5kdc/test-user.keytab.
Aug 12 02:36:35 dist-test-slave-mn4r krb5kdc[12699](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502195, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T02:36:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T02:36:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.132.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
I20260812 02:36:35.687201 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502156971023-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502156971023-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502156971023-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:41403
--webserver_interface=127.7.132.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:45963
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.132.190:41403
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:36:36.025859 12714 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:36:36.026192 12714 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:36:36.026371 12714 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:36:36.033639 12714 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:36:36.033748 12714 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:36:36.033840 12714 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:36:36.033951 12714 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:36:36.034039 12714 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:36:36.044027 12714 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:45963
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502156971023-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502156971023-7698-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.132.190:41403
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:41403
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.132.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502156971023-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:36:36.045462 12714 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:36:36.046870 12714 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:36:36.058687 12723 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:36:36.058759 12720 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:36:36.058794 12721 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:36:36.059880 12714 server_base.cc:1034] running on GCE node
I20260812 02:36:36.060680 12714 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:36:36.062285 12714 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:36:36.063498 12714 hybrid_clock.cc:648] HybridClock initialized: now 1786502196063429 us; error 66 us; skew 500 ppm
I20260812 02:36:36.063922 12714 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:36:36 dist-test-slave-mn4r krb5kdc[12699](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502196, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:36:36.067466 12714 init.cc:377] Logged in from keytab as kudu/127.7.132.190@KRBTEST.COM (short username kudu)
I20260812 02:36:36.068804 12714 fs_manager.cc:359] Metadata directory not provided
I20260812 02:36:36.068903 12714 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:36:36.069252 12714 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:36:36.072547 12714 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502156971023-7698-0/minicluster-data/master-0/data/instance:
uuid: "b69b5c82025242658e1cedf42a6b3440"
format_stamp: "Formatted at 2026-08-12 02:36:36 on dist-test-slave-mn4r"
server_key: "c9ecfd38f0d74e3a44bf5fa4cf3f6cde"
server_key_iv: "fce5cf93e1327cf6818ef16db0a45ac1"
server_key_version: "encryptionkey@0"
I20260812 02:36:36.073427 12714 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502156971023-7698-0/minicluster-data/master-0/wal/instance:
uuid: "b69b5c82025242658e1cedf42a6b3440"
format_stamp: "Formatted at 2026-08-12 02:36:36 on dist-test-slave-mn4r"
server_key: "c9ecfd38f0d74e3a44bf5fa4cf3f6cde"
server_key_iv: "fce5cf93e1327cf6818ef16db0a45ac1"
server_key_version: "encryptionkey@0"
I20260812 02:36:36.079453 12714 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.006s
I20260812 02:36:36.083362 12729 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:36:36.084793 12714 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 02:36:36.085005 12714 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502156971023-7698-0/minicluster-data/master-0/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502156971023-7698-0/minicluster-data/master-0/wal
uuid: "b69b5c82025242658e1cedf42a6b3440"
format_stamp: "Formatted at 2026-08-12 02:36:36 on dist-test-slave-mn4r"
server_key: "c9ecfd38f0d74e3a44bf5fa4cf3f6cde"
server_key_iv: "fce5cf93e1327cf6818ef16db0a45ac1"
server_key_version: "encryptionkey@0"
I20260812 02:36:36.085273 12714 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502156971023-7698-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502156971023-7698-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502156971023-7698-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:36:36.105446 12714 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:36:36.110247 12714 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:36:36.110549 12714 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:36:36.129938 12714 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.190:41403
I20260812 02:36:36.129959 12791 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.190:41403 every 8 connection(s)
I20260812 02:36:36.130957 12714 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
I20260812 02:36:36.135082 12792 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:36:36.141180 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 12714
I20260812 02:36:36.141435 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502156971023-7698-0/minicluster-data/master-0/wal/instance
I20260812 02:36:36.142104 7698 external_mini_cluster.cc:1428] Setting key e3c6d712dafd64106e95758ee51546f4
I20260812 02:36:36.147481 12792 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b69b5c82025242658e1cedf42a6b3440: Bootstrap starting.
I20260812 02:36:36.151813 12792 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b69b5c82025242658e1cedf42a6b3440: Neither blocks nor log segments found. Creating new log.
I20260812 02:36:36.153239 12792 log.cc:824] T 00000000000000000000000000000000 P b69b5c82025242658e1cedf42a6b3440: Log is configured to *not* fsync() on all Append() calls
I20260812 02:36:36.157222 12792 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b69b5c82025242658e1cedf42a6b3440: No bootstrap required, opened a new log
Aug 12 02:36:36 dist-test-slave-mn4r krb5kdc[12699](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502195, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 02:36:36.166721 12792 raft_consensus.cc:348] T 00000000000000000000000000000000 P b69b5c82025242658e1cedf42a6b3440 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b69b5c82025242658e1cedf42a6b3440" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 41403 } }
I20260812 02:36:36.167268 12792 raft_consensus.cc:374] T 00000000000000000000000000000000 P b69b5c82025242658e1cedf42a6b3440 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:36:36.167418 12792 raft_consensus.cc:729] T 00000000000000000000000000000000 P b69b5c82025242658e1cedf42a6b3440 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b69b5c82025242658e1cedf42a6b3440, State: Initialized, Role: FOLLOWER
I20260812 02:36:36.168102 12792 consensus_queue.cc:260] T 00000000000000000000000000000000 P b69b5c82025242658e1cedf42a6b3440 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b69b5c82025242658e1cedf42a6b3440" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 41403 } }
I20260812 02:36:36.168363 12792 raft_consensus.cc:388] T 00000000000000000000000000000000 P b69b5c82025242658e1cedf42a6b3440 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 02:36:36.168409 12795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:36.144272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47936 (local address 127.7.132.190:41403)
0812 02:36:36.145232 (+ 960us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:36.145245 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:36.145293 (+ 48us) server_negotiation.cc:407] Connection header received
0812 02:36:36.147071 (+ 1778us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:36.147100 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:36.147504 (+ 404us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:36.147901 (+ 397us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:36.150187 (+ 2286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:36.151534 (+ 1347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:36.153577 (+ 2043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:36.154725 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:36.154898 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:36.160175 (+ 5277us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:36.160188 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:36.160207 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:36.160249 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:36.163732 (+ 3483us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:36.164494 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:36.164501 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:36.164512 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:36.164716 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:36.165157 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:36.165162 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:36.165164 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:36.165771 (+ 607us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:36.166136 (+ 365us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:36.166604 (+ 468us) server_negotiation.cc:299] Negotiation successful
0812 02:36:36.167050 (+ 446us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":726,"thread_start_us":431,"threads_started":1}
I20260812 02:36:36.168532 12792 raft_consensus.cc:482] T 00000000000000000000000000000000 P b69b5c82025242658e1cedf42a6b3440 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 02:36:36.168728 12792 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b69b5c82025242658e1cedf42a6b3440 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:36:36.171653 12792 raft_consensus.cc:504] T 00000000000000000000000000000000 P b69b5c82025242658e1cedf42a6b3440 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b69b5c82025242658e1cedf42a6b3440" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 41403 } }
I20260812 02:36:36.172214 12792 leader_election.cc:302] T 00000000000000000000000000000000 P b69b5c82025242658e1cedf42a6b3440 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b69b5c82025242658e1cedf42a6b3440; no voters:
I20260812 02:36:36.172955 12792 leader_election.cc:288] T 00000000000000000000000000000000 P b69b5c82025242658e1cedf42a6b3440 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 02:36:36.173024 12797 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b69b5c82025242658e1cedf42a6b3440 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:36:36.174454 12797 raft_consensus.cc:686] T 00000000000000000000000000000000 P b69b5c82025242658e1cedf42a6b3440 [term 1 LEADER]: Becoming Leader. State: Replica: b69b5c82025242658e1cedf42a6b3440, State: Running, Role: LEADER
I20260812 02:36:36.175163 12797 consensus_queue.cc:237] T 00000000000000000000000000000000 P b69b5c82025242658e1cedf42a6b3440 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b69b5c82025242658e1cedf42a6b3440" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 41403 } }
I20260812 02:36:36.175762 12792 sys_catalog.cc:564] T 00000000000000000000000000000000 P b69b5c82025242658e1cedf42a6b3440 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:36:36.178103 12800 sys_catalog.cc:455] T 00000000000000000000000000000000 P b69b5c82025242658e1cedf42a6b3440 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b69b5c82025242658e1cedf42a6b3440. Latest consensus state: current_term: 1 leader_uuid: "b69b5c82025242658e1cedf42a6b3440" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b69b5c82025242658e1cedf42a6b3440" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 41403 } } }
I20260812 02:36:36.178504 12800 sys_catalog.cc:458] T 00000000000000000000000000000000 P b69b5c82025242658e1cedf42a6b3440 [sys.catalog]: This master's current role is: LEADER
I20260812 02:36:36.179420 12805 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:36:36.179502 12798 sys_catalog.cc:455] T 00000000000000000000000000000000 P b69b5c82025242658e1cedf42a6b3440 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b69b5c82025242658e1cedf42a6b3440" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b69b5c82025242658e1cedf42a6b3440" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 41403 } } }
I20260812 02:36:36.179834 12798 sys_catalog.cc:458] T 00000000000000000000000000000000 P b69b5c82025242658e1cedf42a6b3440 [sys.catalog]: This master's current role is: LEADER
I20260812 02:36:36.186327 12805 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 02:36:36.196173 12805 catalog_manager.cc:1347] Generated new cluster ID: b2adead500f14219ab153a0d86a3152e
I20260812 02:36:36.196302 12805 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 02:36:36.231069 12805 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 02:36:36.232471 12805 catalog_manager.cc:1504] Loading token signing keys...
I20260812 02:36:36.238651 12805 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b69b5c82025242658e1cedf42a6b3440: Generated new TSK 0
I20260812 02:36:36.239498 12805 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.132.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.129.keytab.
Entry for principal kudu/127.7.132.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.129.keytab.
I20260812 02:36:36.298143 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502156971023-7698-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502156971023-7698-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502156971023-7698-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502156971023-7698-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.129:0
--local_ip_for_outbound_sockets=127.7.132.129
--webserver_interface=127.7.132.129
--webserver_port=0
--tserver_master_addrs=127.7.132.190:41403
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.129.keytab
--principal=kudu/127.7.132.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:45963
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:36:36.642040 12822 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:36:36.642407 12822 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:36:36.642561 12822 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:36:36.649772 12822 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:36:36.649878 12822 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:36:36.650070 12822 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.129
I20260812 02:36:36.660169 12822 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:45963
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502156971023-7698-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502156971023-7698-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.129.keytab
--principal=kudu/127.7.132.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502156971023-7698-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.132.129
--webserver_port=0
--tserver_master_addrs=127.7.132.190:41403
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.129
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502156971023-7698-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:36:36.661530 12822 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:36:36.663074 12822 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:36:36.677639 12830 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:36:36.677563 12835 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:36:36.677605 12831 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:36:36.678571 12822 server_base.cc:1034] running on GCE node
I20260812 02:36:36.679250 12822 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:36:36.680390 12822 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:36:36.681659 12822 hybrid_clock.cc:648] HybridClock initialized: now 1786502196681611 us; error 46 us; skew 500 ppm
I20260812 02:36:36.682024 12822 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:36:36 dist-test-slave-mn4r krb5kdc[12699](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502196, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:36:36.685429 12822 init.cc:377] Logged in from keytab as kudu/127.7.132.129@KRBTEST.COM (short username kudu)
I20260812 02:36:36.686719 12822 fs_manager.cc:359] Metadata directory not provided
I20260812 02:36:36.686861 12822 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:36:36.687197 12822 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:36:36.690476 12822 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502156971023-7698-0/minicluster-data/ts-0/data/instance:
uuid: "eaeef5241eb24a8eaa0b92e919c96b8b"
format_stamp: "Formatted at 2026-08-12 02:36:36 on dist-test-slave-mn4r"
server_key: "d4f482d94d93f1588fda54efce11d80c"
server_key_iv: "86aa097aef4b283fa31ba80bf23dcf9d"
server_key_version: "encryptionkey@0"
I20260812 02:36:36.691323 12822 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502156971023-7698-0/minicluster-data/ts-0/wal/instance:
uuid: "eaeef5241eb24a8eaa0b92e919c96b8b"
format_stamp: "Formatted at 2026-08-12 02:36:36 on dist-test-slave-mn4r"
server_key: "d4f482d94d93f1588fda54efce11d80c"
server_key_iv: "86aa097aef4b283fa31ba80bf23dcf9d"
server_key_version: "encryptionkey@0"
I20260812 02:36:36.696712 12822 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260812 02:36:36.700332 12843 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:36:36.701779 12822 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 02:36:36.702008 12822 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502156971023-7698-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502156971023-7698-0/minicluster-data/ts-0/wal
uuid: "eaeef5241eb24a8eaa0b92e919c96b8b"
format_stamp: "Formatted at 2026-08-12 02:36:36 on dist-test-slave-mn4r"
server_key: "d4f482d94d93f1588fda54efce11d80c"
server_key_iv: "86aa097aef4b283fa31ba80bf23dcf9d"
server_key_version: "encryptionkey@0"
I20260812 02:36:36.702296 12822 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502156971023-7698-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502156971023-7698-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502156971023-7698-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:36:36.720408 12822 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:36:36.724993 12822 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:36:36.725301 12822 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:36:36.726462 12822 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:36:36.727880 12822 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:36:36.728018 12822 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:36:36.728158 12822 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:36:36.728252 12822 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:36:36.762677 12822 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.129:39939
I20260812 02:36:36.762732 12978 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.129:39939 every 8 connection(s)
I20260812 02:36:36.763612 12822 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502156971023-7698-0/minicluster-data/ts-0/data/info.pb
I20260812 02:36:36.772665 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 12822
I20260812 02:36:36.772898 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502156971023-7698-0/minicluster-data/ts-0/wal/instance
I20260812 02:36:36.773612 7698 external_mini_cluster.cc:1428] Setting key fedea8f367b9db72a5f07ec5e43bf226
Aug 12 02:36:36 dist-test-slave-mn4r krb5kdc[12699](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502196, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.129@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 02:36:36.789551 12982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:36.766212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:41413 (local address 127.7.132.190:41403)
0812 02:36:36.766658 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:36.766663 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:36.767741 (+ 1078us) server_negotiation.cc:407] Connection header received
0812 02:36:36.769504 (+ 1763us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:36.769508 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:36.769578 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:36.769674 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:36.771213 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:36.771924 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:36.773618 (+ 1694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:36.774233 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:36.774353 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:36.781992 (+ 7639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:36.782001 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:36.782008 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:36.782056 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:36.785812 (+ 3756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:36.786854 (+ 1042us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:36.786858 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:36.786860 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:36.786992 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:36.787487 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:36.787491 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:36.787492 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:36.787755 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:36.787863 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:36.788971 (+ 1108us) server_negotiation.cc:299] Negotiation successful
0812 02:36:36.789162 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":203,"threads_started":1}
I20260812 02:36:36.791081 12983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:36.766814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:41403 (local address 127.7.132.129:41413)
0812 02:36:36.767548 (+ 734us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:36.767599 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:36.769242 (+ 1643us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:36.769886 (+ 644us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:36.769894 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:36.770493 (+ 599us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:36:36.771054 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:36.771072 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:36.772096 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:36.772099 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:36.773458 (+ 1359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:36.773471 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:36.774510 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:36.774519 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:36.774741 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:36.778593 (+ 3852us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:36:36.778635 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:36:36.781714 (+ 3079us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:36:36.785973 (+ 4259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:36.785985 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:36.786007 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:36.786723 (+ 716us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:36.787108 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:36.787115 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:36.787121 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:36.787368 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:36.787890 (+ 522us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:36:36.787901 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:36:36.788413 (+ 512us) client_negotiation.cc:769] Sending connection context
0812 02:36:36.788815 (+ 402us) client_negotiation.cc:240] Negotiation successful
0812 02:36:36.789469 (+ 654us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":293,"threads_started":1}
I20260812 02:36:36.792842 12979 heartbeater.cc:347] Connected to a master server at 127.7.132.190:41403
I20260812 02:36:36.793190 12979 heartbeater.cc:464] Registering TS with master...
I20260812 02:36:36.794152 12979 heartbeater.cc:511] Master 127.7.132.190:41403 requested a full tablet report, sending...
I20260812 02:36:36.796514 12750 ts_manager.cc:194] Registered new tserver with Master: eaeef5241eb24a8eaa0b92e919c96b8b (127.7.132.129:39939)
I20260812 02:36:36.798307 12750 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.129@KRBTEST.COM'} at 127.7.132.129:41413
WARNING: no policy specified for kudu/127.7.132.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.130.keytab.
Entry for principal kudu/127.7.132.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.130.keytab.
I20260812 02:36:36.829317 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502156971023-7698-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502156971023-7698-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502156971023-7698-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502156971023-7698-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.130:0
--local_ip_for_outbound_sockets=127.7.132.130
--webserver_interface=127.7.132.130
--webserver_port=0
--tserver_master_addrs=127.7.132.190:41403
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.130.keytab
--principal=kudu/127.7.132.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:45963
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:36:37.170779 12989 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:36:37.171078 12989 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:36:37.171281 12989 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:36:37.178906 12989 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:36:37.179051 12989 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:36:37.179244 12989 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.130
I20260812 02:36:37.189906 12989 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:45963
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502156971023-7698-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502156971023-7698-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.130.keytab
--principal=kudu/127.7.132.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502156971023-7698-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.132.130
--webserver_port=0
--tserver_master_addrs=127.7.132.190:41403
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.130
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502156971023-7698-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:36:37.191293 12989 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:36:37.192669 12989 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:36:37.206583 12996 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:36:37.206687 12995 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:36:37.206553 12998 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:36:37.207517 12989 server_base.cc:1034] running on GCE node
I20260812 02:36:37.208073 12989 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:36:37.209017 12989 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:36:37.210202 12989 hybrid_clock.cc:648] HybridClock initialized: now 1786502197210138 us; error 45 us; skew 500 ppm
I20260812 02:36:37.210508 12989 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:36:37 dist-test-slave-mn4r krb5kdc[12699](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502197, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:36:37.213680 12989 init.cc:377] Logged in from keytab as kudu/127.7.132.130@KRBTEST.COM (short username kudu)
I20260812 02:36:37.214881 12989 fs_manager.cc:359] Metadata directory not provided
I20260812 02:36:37.214996 12989 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:36:37.215338 12989 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:36:37.218350 12989 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502156971023-7698-0/minicluster-data/ts-1/data/instance:
uuid: "e2001fa695c0411996e682951e2f7307"
format_stamp: "Formatted at 2026-08-12 02:36:37 on dist-test-slave-mn4r"
server_key: "2c524b8ff7622fcb598552628fc5ec81"
server_key_iv: "958b4743d1295593e7c4083bcaed860c"
server_key_version: "encryptionkey@0"
I20260812 02:36:37.219210 12989 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502156971023-7698-0/minicluster-data/ts-1/wal/instance:
uuid: "e2001fa695c0411996e682951e2f7307"
format_stamp: "Formatted at 2026-08-12 02:36:37 on dist-test-slave-mn4r"
server_key: "2c524b8ff7622fcb598552628fc5ec81"
server_key_iv: "958b4743d1295593e7c4083bcaed860c"
server_key_version: "encryptionkey@0"
I20260812 02:36:37.224309 12989 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260812 02:36:37.227831 13005 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:36:37.229192 12989 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 02:36:37.229358 12989 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502156971023-7698-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502156971023-7698-0/minicluster-data/ts-1/wal
uuid: "e2001fa695c0411996e682951e2f7307"
format_stamp: "Formatted at 2026-08-12 02:36:37 on dist-test-slave-mn4r"
server_key: "2c524b8ff7622fcb598552628fc5ec81"
server_key_iv: "958b4743d1295593e7c4083bcaed860c"
server_key_version: "encryptionkey@0"
I20260812 02:36:37.229578 12989 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502156971023-7698-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502156971023-7698-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502156971023-7698-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:36:37.248874 12989 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:36:37.253273 12989 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:36:37.253585 12989 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:36:37.254611 12989 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:36:37.256225 12989 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:36:37.256362 12989 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:36:37.256520 12989 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:36:37.256609 12989 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:36:37.289314 12989 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.130:37419
I20260812 02:36:37.289359 13137 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.130:37419 every 8 connection(s)
I20260812 02:36:37.290517 12989 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502156971023-7698-0/minicluster-data/ts-1/data/info.pb
I20260812 02:36:37.291829 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 12989
I20260812 02:36:37.292050 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502156971023-7698-0/minicluster-data/ts-1/wal/instance
I20260812 02:36:37.292505 7698 external_mini_cluster.cc:1428] Setting key 067861a5dd4805e173af7848a5efc6ab
Aug 12 02:36:37 dist-test-slave-mn4r krb5kdc[12699](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502197, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.130@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 02:36:37.311502 13140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:37.293568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:60119 (local address 127.7.132.190:41403)
0812 02:36:37.294005 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:37.294010 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:37.295364 (+ 1354us) server_negotiation.cc:407] Connection header received
0812 02:36:37.297201 (+ 1837us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:37.297207 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:37.297300 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:37.297431 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:37.299081 (+ 1650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:37.299775 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:37.301324 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:37.301770 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:37.301924 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:37.305489 (+ 3565us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:37.305494 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:37.305497 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:37.305530 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:37.308345 (+ 2815us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:37.309103 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:37.309113 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:37.309116 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:37.309237 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:37.309682 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:37.309686 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:37.309688 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:37.309952 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:37.310161 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:37.310870 (+ 709us) server_negotiation.cc:299] Negotiation successful
0812 02:36:37.311107 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":212,"threads_started":1}
I20260812 02:36:37.312490 13141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:37.294277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:41403 (local address 127.7.132.130:60119)
0812 02:36:37.295208 (+ 931us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:37.295262 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:37.296931 (+ 1669us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:37.297669 (+ 738us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:37.297677 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:37.298303 (+ 626us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:36:37.298881 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:37.298913 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:37.299938 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:37.299942 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:37.301175 (+ 1233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:37.301187 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:37.302025 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:37.302032 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:37.302287 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:37.302993 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:36:37.303015 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:36:37.305258 (+ 2243us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:36:37.308499 (+ 3241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:37.308508 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:37.308523 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:37.308969 (+ 446us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:37.309350 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:37.309355 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:37.309359 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:37.309552 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:37.310161 (+ 609us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:36:37.310169 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:36:37.310505 (+ 336us) client_negotiation.cc:769] Sending connection context
0812 02:36:37.310775 (+ 270us) client_negotiation.cc:240] Negotiation successful
0812 02:36:37.311348 (+ 573us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":560,"thread_start_us":314,"threads_started":1}
I20260812 02:36:37.313984 13138 heartbeater.cc:347] Connected to a master server at 127.7.132.190:41403
I20260812 02:36:37.314332 13138 heartbeater.cc:464] Registering TS with master...
I20260812 02:36:37.314990 13138 heartbeater.cc:511] Master 127.7.132.190:41403 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.7.132.131@KRBTEST.COM; defaulting to no policy
I20260812 02:36:37.316504 12750 ts_manager.cc:194] Registered new tserver with Master: e2001fa695c0411996e682951e2f7307 (127.7.132.130:37419)
I20260812 02:36:37.317348 12750 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.130@KRBTEST.COM'} at 127.7.132.130:60119
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.131.keytab.
Entry for principal kudu/127.7.132.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.131.keytab.
I20260812 02:36:37.346217 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502156971023-7698-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502156971023-7698-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502156971023-7698-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502156971023-7698-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.131:0
--local_ip_for_outbound_sockets=127.7.132.131
--webserver_interface=127.7.132.131
--webserver_port=0
--tserver_master_addrs=127.7.132.190:41403
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.131.keytab
--principal=kudu/127.7.132.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:45963
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:36:37.735122 13144 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:36:37.735409 13144 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:36:37.735608 13144 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:36:37.742879 13144 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:36:37.743007 13144 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:36:37.743193 13144 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.131
I20260812 02:36:37.752861 13144 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:45963
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502156971023-7698-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502156971023-7698-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.131.keytab
--principal=kudu/127.7.132.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502156971023-7698-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.132.131
--webserver_port=0
--tserver_master_addrs=127.7.132.190:41403
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.131
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502156971023-7698-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:36:37.754186 13144 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:36:37.755558 13144 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:36:37.769068 13151 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:36:37.769227 13154 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:36:37.769079 13150 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:36:37.769833 13144 server_base.cc:1034] running on GCE node
I20260812 02:36:37.770553 13144 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:36:37.771483 13144 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:36:37.772778 13144 hybrid_clock.cc:648] HybridClock initialized: now 1786502197772726 us; error 59 us; skew 500 ppm
I20260812 02:36:37.773144 13144 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:36:37 dist-test-slave-mn4r krb5kdc[12699](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502197, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:36:37.776393 13144 init.cc:377] Logged in from keytab as kudu/127.7.132.131@KRBTEST.COM (short username kudu)
I20260812 02:36:37.777695 13144 fs_manager.cc:359] Metadata directory not provided
I20260812 02:36:37.777823 13144 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:36:37.778239 13144 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:36:37.781735 13144 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502156971023-7698-0/minicluster-data/ts-2/data/instance:
uuid: "462d914ce8f14ee9af2b267aa392f1b4"
format_stamp: "Formatted at 2026-08-12 02:36:37 on dist-test-slave-mn4r"
server_key: "cc574384ab2f47c1a260375b471bb30d"
server_key_iv: "92740d159bcd576f3ffd19fe393bdd9c"
server_key_version: "encryptionkey@0"
I20260812 02:36:37.782662 13144 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502156971023-7698-0/minicluster-data/ts-2/wal/instance:
uuid: "462d914ce8f14ee9af2b267aa392f1b4"
format_stamp: "Formatted at 2026-08-12 02:36:37 on dist-test-slave-mn4r"
server_key: "cc574384ab2f47c1a260375b471bb30d"
server_key_iv: "92740d159bcd576f3ffd19fe393bdd9c"
server_key_version: "encryptionkey@0"
I20260812 02:36:37.788209 13144 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 02:36:37.792088 13160 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:36:37.793464 13144 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 02:36:37.793648 13144 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502156971023-7698-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502156971023-7698-0/minicluster-data/ts-2/wal
uuid: "462d914ce8f14ee9af2b267aa392f1b4"
format_stamp: "Formatted at 2026-08-12 02:36:37 on dist-test-slave-mn4r"
server_key: "cc574384ab2f47c1a260375b471bb30d"
server_key_iv: "92740d159bcd576f3ffd19fe393bdd9c"
server_key_version: "encryptionkey@0"
I20260812 02:36:37.793895 13144 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502156971023-7698-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502156971023-7698-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502156971023-7698-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:36:37.801357 12979 heartbeater.cc:503] Master 127.7.132.190:41403 was elected leader, sending a full tablet report...
I20260812 02:36:37.821563 13144 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:36:37.825732 13144 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:36:37.826047 13144 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:36:37.827112 13144 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:36:37.828469 13144 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:36:37.828595 13144 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:36:37.828733 13144 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:36:37.828824 13144 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:36:37.862581 13144 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.131:39419
I20260812 02:36:37.862668 13297 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.131:39419 every 8 connection(s)
I20260812 02:36:37.863493 13144 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502156971023-7698-0/minicluster-data/ts-2/data/info.pb
I20260812 02:36:37.871142 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 13144
I20260812 02:36:37.871371 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502156971023-7698-0/minicluster-data/ts-2/wal/instance
I20260812 02:36:37.871920 7698 external_mini_cluster.cc:1428] Setting key e67d69ae81056deb884a1d716d319927
Aug 12 02:36:37 dist-test-slave-mn4r krb5kdc[12699](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502197, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.131@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 02:36:37.884773 13300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:37.866200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:46393 (local address 127.7.132.190:41403)
0812 02:36:37.866657 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:37.866662 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:37.867709 (+ 1047us) server_negotiation.cc:407] Connection header received
0812 02:36:37.869569 (+ 1860us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:37.869574 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:37.869677 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:37.869803 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:37.871724 (+ 1921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:37.872596 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:37.874708 (+ 2112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:37.875286 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:37.875423 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:37.879438 (+ 4015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:37.879443 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:37.879447 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:37.879479 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:37.881911 (+ 2432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:37.882677 (+ 766us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:37.882681 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:37.882683 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:37.882799 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:37.883227 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:37.883231 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:37.883233 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:37.883536 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:37.883656 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:37.884324 (+ 668us) server_negotiation.cc:299] Negotiation successful
0812 02:36:37.884494 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":338,"thread_start_us":230,"threads_started":1}
I20260812 02:36:37.885864 13301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:37.866815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:41403 (local address 127.7.132.131:46393)
0812 02:36:37.867541 (+ 726us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:37.867588 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:37.869297 (+ 1709us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:37.870046 (+ 749us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:37.870054 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:37.870624 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:36:37.871523 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:37.871554 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:37.872767 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:37.872773 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:37.874560 (+ 1787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:37.874572 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:37.875558 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:37.875566 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:37.875812 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:37.876626 (+ 814us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:36:37.876659 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:36:37.879209 (+ 2550us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:36:37.882038 (+ 2829us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:37.882045 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:37.882056 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:37.882562 (+ 506us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:37.882893 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:37.882896 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:37.882898 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:37.883107 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:37.883657 (+ 550us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:36:37.883662 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:36:37.883973 (+ 311us) client_negotiation.cc:769] Sending connection context
0812 02:36:37.884193 (+ 220us) client_negotiation.cc:240] Negotiation successful
0812 02:36:37.884584 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":488,"thread_start_us":314,"threads_started":1}
I20260812 02:36:37.887544 13298 heartbeater.cc:347] Connected to a master server at 127.7.132.190:41403
I20260812 02:36:37.887857 13298 heartbeater.cc:464] Registering TS with master...
I20260812 02:36:37.888505 13298 heartbeater.cc:511] Master 127.7.132.190:41403 requested a full tablet report, sending...
I20260812 02:36:37.890175 12748 ts_manager.cc:194] Registered new tserver with Master: 462d914ce8f14ee9af2b267aa392f1b4 (127.7.132.131:39419)
I20260812 02:36:37.891122 12748 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.131@KRBTEST.COM'} at 127.7.132.131:46393
I20260812 02:36:37.899549 7698 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 02:36:37.916360 13300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:37.904107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52642 (local address 127.7.132.190:41403)
0812 02:36:37.904312 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:37.904321 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:37.904770 (+ 449us) server_negotiation.cc:407] Connection header received
0812 02:36:37.904898 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:37.904902 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:37.904964 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:37.905044 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:37.906570 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:37.907366 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:37.908571 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:37.908986 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:37.909078 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:37.911256 (+ 2178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:37.911261 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:37.911264 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:37.911295 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:37.913872 (+ 2577us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:37.914591 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:37.914595 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:37.914597 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:37.914728 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:37.915074 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:37.915077 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:37.915079 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:37.915384 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:37.915556 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:37.915847 (+ 291us) server_negotiation.cc:299] Negotiation successful
0812 02:36:37.916017 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 02:36:37.920382 12748 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52642:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 02:36:37.927222 12748 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 02:36:37.956926 13314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:37.939905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34070 (local address 127.7.132.130:37419)
0812 02:36:37.940985 (+ 1080us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:37.940997 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:37.941030 (+ 33us) server_negotiation.cc:407] Connection header received
0812 02:36:37.941168 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:37.941176 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:37.941484 (+ 308us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:37.941751 (+ 267us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:36:37.944423 (+ 2672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:37.948462 (+ 4039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:37.953973 (+ 5511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:37.955600 (+ 1627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:37.955764 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:37.956021 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:37.956412 (+ 391us) server_negotiation.cc:299] Negotiation successful
0812 02:36:37.956594 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":908,"thread_start_us":1115,"threads_started":1}
I20260812 02:36:37.956962 13313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:37.939881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.130:37419 (local address 127.0.0.1:34070)
0812 02:36:37.940748 (+ 867us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:37.940787 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:37.940964 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:37.942142 (+ 1178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:37.942153 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:37.942200 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:36:37.944155 (+ 1955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:37.944186 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:37.948691 (+ 4505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:37.948700 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:37.953712 (+ 5012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:37.953729 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:37.955884 (+ 2155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:37.955891 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:37.956160 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:37.956225 (+ 65us) client_negotiation.cc:769] Sending connection context
0812 02:36:37.956358 (+ 133us) client_negotiation.cc:240] Negotiation successful
0812 02:36:37.956570 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":597,"thread_start_us":714,"threads_started":1}
I20260812 02:36:37.962275 13315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:37.940415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.131:39419 (local address 127.0.0.1:46856)
0812 02:36:37.953600 (+ 13185us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:37.953625 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:37.953783 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:37.954658 (+ 875us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:37.954667 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:37.954700 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:36:37.955139 (+ 439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:37.955150 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:37.957318 (+ 2168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:37.957323 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:37.959868 (+ 2545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:37.959886 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:37.961443 (+ 1557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:37.961449 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:37.961610 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:37.961643 (+ 33us) client_negotiation.cc:769] Sending connection context
0812 02:36:37.961749 (+ 106us) client_negotiation.cc:240] Negotiation successful
0812 02:36:37.961907 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13012,"thread_start_us":12900,"threads_started":1}
I20260812 02:36:37.962437 13316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:37.940760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46856 (local address 127.7.132.131:39419)
0812 02:36:37.953289 (+ 12529us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:37.953302 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:37.953931 (+ 629us) server_negotiation.cc:407] Connection header received
0812 02:36:37.954053 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:37.954066 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:37.954412 (+ 346us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:37.954654 (+ 242us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:36:37.955365 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:37.957143 (+ 1778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:37.960059 (+ 2916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:37.961222 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:37.961367 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:37.961635 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:37.961809 (+ 174us) server_negotiation.cc:299] Negotiation successful
0812 02:36:37.962009 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12296,"thread_start_us":12080,"threads_started":1}
I20260812 02:36:37.961745 13058 tablet_service.cc:1467] Processing CreateTablet for tablet 7b14e504553b46c7bdfc8a5ae828477f (DEFAULT_TABLE table=test-table [id=4e12075f15194bf689bdbf30440a88d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:36:37.963812 13058 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b14e504553b46c7bdfc8a5ae828477f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:36:37.965204 13317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:37.941385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:39939 (local address 127.0.0.1:37634)
0812 02:36:37.954629 (+ 13244us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:37.954647 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:37.954771 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:37.955541 (+ 770us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:37.955545 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:37.955571 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:36:37.955997 (+ 426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:37.956015 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:37.958690 (+ 2675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:37.958695 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:37.960846 (+ 2151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:37.960861 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:37.964411 (+ 3550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:37.964419 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:37.964593 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:37.964625 (+ 32us) client_negotiation.cc:769] Sending connection context
0812 02:36:37.964726 (+ 101us) client_negotiation.cc:240] Negotiation successful
0812 02:36:37.964849 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13079,"thread_start_us":12956,"threads_started":1}
I20260812 02:36:37.965472 13318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:37.941985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37634 (local address 127.7.132.129:39939)
0812 02:36:37.954145 (+ 12160us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:37.954158 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:37.954931 (+ 773us) server_negotiation.cc:407] Connection header received
0812 02:36:37.955062 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:37.955072 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:37.955344 (+ 272us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:37.955539 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:36:37.956211 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:37.958444 (+ 2233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:37.961044 (+ 2600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:37.963901 (+ 2857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:37.964051 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:37.964271 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:37.964811 (+ 540us) server_negotiation.cc:299] Negotiation successful
0812 02:36:37.964999 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11904,"thread_start_us":11744,"threads_started":1}
I20260812 02:36:37.966504 13218 tablet_service.cc:1467] Processing CreateTablet for tablet 7b14e504553b46c7bdfc8a5ae828477f (DEFAULT_TABLE table=test-table [id=4e12075f15194bf689bdbf30440a88d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:36:37.968129 13218 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b14e504553b46c7bdfc8a5ae828477f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:36:37.969635 12900 tablet_service.cc:1467] Processing CreateTablet for tablet 7b14e504553b46c7bdfc8a5ae828477f (DEFAULT_TABLE table=test-table [id=4e12075f15194bf689bdbf30440a88d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:36:37.971715 12900 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b14e504553b46c7bdfc8a5ae828477f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:36:37.976022 13319 tablet_bootstrap.cc:492] T 7b14e504553b46c7bdfc8a5ae828477f P e2001fa695c0411996e682951e2f7307: Bootstrap starting.
I20260812 02:36:37.979715 13319 tablet_bootstrap.cc:654] T 7b14e504553b46c7bdfc8a5ae828477f P e2001fa695c0411996e682951e2f7307: Neither blocks nor log segments found. Creating new log.
I20260812 02:36:37.980911 13319 log.cc:824] T 7b14e504553b46c7bdfc8a5ae828477f P e2001fa695c0411996e682951e2f7307: Log is configured to *not* fsync() on all Append() calls
I20260812 02:36:37.987035 13323 tablet_bootstrap.cc:492] T 7b14e504553b46c7bdfc8a5ae828477f P 462d914ce8f14ee9af2b267aa392f1b4: Bootstrap starting.
I20260812 02:36:37.990180 13319 tablet_bootstrap.cc:492] T 7b14e504553b46c7bdfc8a5ae828477f P e2001fa695c0411996e682951e2f7307: No bootstrap required, opened a new log
I20260812 02:36:37.990584 13319 ts_tablet_manager.cc:1397] T 7b14e504553b46c7bdfc8a5ae828477f P e2001fa695c0411996e682951e2f7307: Time spent bootstrapping tablet: real 0.015s user 0.010s sys 0.004s
I20260812 02:36:37.990878 13324 tablet_bootstrap.cc:492] T 7b14e504553b46c7bdfc8a5ae828477f P eaeef5241eb24a8eaa0b92e919c96b8b: Bootstrap starting.
I20260812 02:36:37.991717 13323 tablet_bootstrap.cc:654] T 7b14e504553b46c7bdfc8a5ae828477f P 462d914ce8f14ee9af2b267aa392f1b4: Neither blocks nor log segments found. Creating new log.
I20260812 02:36:37.992848 13323 log.cc:824] T 7b14e504553b46c7bdfc8a5ae828477f P 462d914ce8f14ee9af2b267aa392f1b4: Log is configured to *not* fsync() on all Append() calls
I20260812 02:36:37.994422 13324 tablet_bootstrap.cc:654] T 7b14e504553b46c7bdfc8a5ae828477f P eaeef5241eb24a8eaa0b92e919c96b8b: Neither blocks nor log segments found. Creating new log.
I20260812 02:36:37.995682 13324 log.cc:824] T 7b14e504553b46c7bdfc8a5ae828477f P eaeef5241eb24a8eaa0b92e919c96b8b: Log is configured to *not* fsync() on all Append() calls
I20260812 02:36:37.995821 13323 tablet_bootstrap.cc:492] T 7b14e504553b46c7bdfc8a5ae828477f P 462d914ce8f14ee9af2b267aa392f1b4: No bootstrap required, opened a new log
I20260812 02:36:37.996140 13323 ts_tablet_manager.cc:1397] T 7b14e504553b46c7bdfc8a5ae828477f P 462d914ce8f14ee9af2b267aa392f1b4: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260812 02:36:37.999015 13324 tablet_bootstrap.cc:492] T 7b14e504553b46c7bdfc8a5ae828477f P eaeef5241eb24a8eaa0b92e919c96b8b: No bootstrap required, opened a new log
I20260812 02:36:37.999289 13324 ts_tablet_manager.cc:1397] T 7b14e504553b46c7bdfc8a5ae828477f P eaeef5241eb24a8eaa0b92e919c96b8b: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260812 02:36:38.001197 13319 raft_consensus.cc:348] T 7b14e504553b46c7bdfc8a5ae828477f P e2001fa695c0411996e682951e2f7307 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2001fa695c0411996e682951e2f7307" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37419 } } peers { permanent_uuid: "eaeef5241eb24a8eaa0b92e919c96b8b" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 39939 } } peers { permanent_uuid: "462d914ce8f14ee9af2b267aa392f1b4" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 39419 } }
I20260812 02:36:38.001824 13319 raft_consensus.cc:374] T 7b14e504553b46c7bdfc8a5ae828477f P e2001fa695c0411996e682951e2f7307 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:36:38.002004 13319 raft_consensus.cc:729] T 7b14e504553b46c7bdfc8a5ae828477f P e2001fa695c0411996e682951e2f7307 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2001fa695c0411996e682951e2f7307, State: Initialized, Role: FOLLOWER
I20260812 02:36:38.002739 13319 consensus_queue.cc:260] T 7b14e504553b46c7bdfc8a5ae828477f P e2001fa695c0411996e682951e2f7307 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2001fa695c0411996e682951e2f7307" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37419 } } peers { permanent_uuid: "eaeef5241eb24a8eaa0b92e919c96b8b" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 39939 } } peers { permanent_uuid: "462d914ce8f14ee9af2b267aa392f1b4" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 39419 } }
I20260812 02:36:38.006211 13323 raft_consensus.cc:348] T 7b14e504553b46c7bdfc8a5ae828477f P 462d914ce8f14ee9af2b267aa392f1b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2001fa695c0411996e682951e2f7307" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37419 } } peers { permanent_uuid: "eaeef5241eb24a8eaa0b92e919c96b8b" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 39939 } } peers { permanent_uuid: "462d914ce8f14ee9af2b267aa392f1b4" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 39419 } }
I20260812 02:36:38.006950 13138 heartbeater.cc:503] Master 127.7.132.190:41403 was elected leader, sending a full tablet report...
I20260812 02:36:38.007622 13319 ts_tablet_manager.cc:1428] T 7b14e504553b46c7bdfc8a5ae828477f P e2001fa695c0411996e682951e2f7307: Time spent starting tablet: real 0.017s user 0.012s sys 0.004s
I20260812 02:36:38.007812 13323 raft_consensus.cc:374] T 7b14e504553b46c7bdfc8a5ae828477f P 462d914ce8f14ee9af2b267aa392f1b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:36:38.007944 13323 raft_consensus.cc:729] T 7b14e504553b46c7bdfc8a5ae828477f P 462d914ce8f14ee9af2b267aa392f1b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 462d914ce8f14ee9af2b267aa392f1b4, State: Initialized, Role: FOLLOWER
I20260812 02:36:38.008586 13323 consensus_queue.cc:260] T 7b14e504553b46c7bdfc8a5ae828477f P 462d914ce8f14ee9af2b267aa392f1b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2001fa695c0411996e682951e2f7307" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37419 } } peers { permanent_uuid: "eaeef5241eb24a8eaa0b92e919c96b8b" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 39939 } } peers { permanent_uuid: "462d914ce8f14ee9af2b267aa392f1b4" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 39419 } }
I20260812 02:36:38.010893 13298 heartbeater.cc:503] Master 127.7.132.190:41403 was elected leader, sending a full tablet report...
I20260812 02:36:38.011590 13323 ts_tablet_manager.cc:1428] T 7b14e504553b46c7bdfc8a5ae828477f P 462d914ce8f14ee9af2b267aa392f1b4: Time spent starting tablet: real 0.015s user 0.013s sys 0.000s
I20260812 02:36:38.011551 13324 raft_consensus.cc:348] T 7b14e504553b46c7bdfc8a5ae828477f P eaeef5241eb24a8eaa0b92e919c96b8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2001fa695c0411996e682951e2f7307" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37419 } } peers { permanent_uuid: "eaeef5241eb24a8eaa0b92e919c96b8b" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 39939 } } peers { permanent_uuid: "462d914ce8f14ee9af2b267aa392f1b4" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 39419 } }
I20260812 02:36:38.012005 13324 raft_consensus.cc:374] T 7b14e504553b46c7bdfc8a5ae828477f P eaeef5241eb24a8eaa0b92e919c96b8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:36:38.012168 13324 raft_consensus.cc:729] T 7b14e504553b46c7bdfc8a5ae828477f P eaeef5241eb24a8eaa0b92e919c96b8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eaeef5241eb24a8eaa0b92e919c96b8b, State: Initialized, Role: FOLLOWER
I20260812 02:36:38.012867 13324 consensus_queue.cc:260] T 7b14e504553b46c7bdfc8a5ae828477f P eaeef5241eb24a8eaa0b92e919c96b8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2001fa695c0411996e682951e2f7307" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37419 } } peers { permanent_uuid: "eaeef5241eb24a8eaa0b92e919c96b8b" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 39939 } } peers { permanent_uuid: "462d914ce8f14ee9af2b267aa392f1b4" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 39419 } }
I20260812 02:36:38.014896 13324 ts_tablet_manager.cc:1428] T 7b14e504553b46c7bdfc8a5ae828477f P eaeef5241eb24a8eaa0b92e919c96b8b: Time spent starting tablet: real 0.015s user 0.015s sys 0.000s
W20260812 02:36:38.015882 12981 tablet.cc:2369] T 7b14e504553b46c7bdfc8a5ae828477f P eaeef5241eb24a8eaa0b92e919c96b8b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 02:36:38.042532 13139 tablet.cc:2369] T 7b14e504553b46c7bdfc8a5ae828477f P e2001fa695c0411996e682951e2f7307: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 02:36:38.115180 13299 tablet.cc:2369] T 7b14e504553b46c7bdfc8a5ae828477f P 462d914ce8f14ee9af2b267aa392f1b4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 02:36:38.248919 13329 raft_consensus.cc:482] T 7b14e504553b46c7bdfc8a5ae828477f P eaeef5241eb24a8eaa0b92e919c96b8b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 02:36:38.249290 13329 raft_consensus.cc:504] T 7b14e504553b46c7bdfc8a5ae828477f P eaeef5241eb24a8eaa0b92e919c96b8b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2001fa695c0411996e682951e2f7307" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37419 } } peers { permanent_uuid: "eaeef5241eb24a8eaa0b92e919c96b8b" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 39939 } } peers { permanent_uuid: "462d914ce8f14ee9af2b267aa392f1b4" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 39419 } }
I20260812 02:36:38.251340 13329 leader_election.cc:288] T 7b14e504553b46c7bdfc8a5ae828477f P eaeef5241eb24a8eaa0b92e919c96b8b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e2001fa695c0411996e682951e2f7307 (127.7.132.130:37419), 462d914ce8f14ee9af2b267aa392f1b4 (127.7.132.131:39419)
I20260812 02:36:38.258523 13316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:38.252000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:40219 (local address 127.7.132.131:39419)
0812 02:36:38.252221 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:38.252230 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:38.252886 (+ 656us) server_negotiation.cc:407] Connection header received
0812 02:36:38.253140 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:38.253145 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:38.253226 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:38.253343 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:36:38.254040 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:38.254873 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:38.256652 (+ 1779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:38.257466 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:38.257593 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:38.257646 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:38.258112 (+ 466us) server_negotiation.cc:299] Negotiation successful
0812 02:36:38.258251 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 02:36:38.258646 13331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:38.251844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.131:39419 (local address 127.7.132.129:40219)
0812 02:36:38.252719 (+ 875us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:38.252738 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:38.252900 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:38.253373 (+ 473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:38.253376 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:38.253396 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:36:38.253843 (+ 447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:38.253858 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:38.255026 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:38.255030 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:38.256492 (+ 1462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:38.256508 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:38.257686 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:38.257693 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:38.257875 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:38.257910 (+ 35us) client_negotiation.cc:769] Sending connection context
0812 02:36:38.258014 (+ 104us) client_negotiation.cc:240] Negotiation successful
0812 02:36:38.258206 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":715,"mutex_wait_us":38,"thread_start_us":475,"threads_started":1}
I20260812 02:36:38.258826 13314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:38.251782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:43233 (local address 127.7.132.130:37419)
0812 02:36:38.252004 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:38.252014 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:38.252697 (+ 683us) server_negotiation.cc:407] Connection header received
0812 02:36:38.252880 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:38.252885 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:38.252975 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:38.253070 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:36:38.253805 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:38.254683 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:38.256717 (+ 2034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:38.257761 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:38.257912 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:38.257995 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:38.258408 (+ 413us) server_negotiation.cc:299] Negotiation successful
0812 02:36:38.258535 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260812 02:36:38.258991 13330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:38.251719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.130:37419 (local address 127.7.132.129:43233)
0812 02:36:38.252563 (+ 844us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:38.252589 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:38.252723 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:38.253139 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:38.253145 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:38.253176 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:36:38.253634 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:38.253646 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:38.254832 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:38.254837 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:38.256524 (+ 1687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:38.256537 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:38.257998 (+ 1461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:38.258005 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:38.258227 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:38.258257 (+ 30us) client_negotiation.cc:769] Sending connection context
0812 02:36:38.258360 (+ 103us) client_negotiation.cc:240] Negotiation successful
0812 02:36:38.258482 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":706,"mutex_wait_us":86,"thread_start_us":516,"threads_started":1}
I20260812 02:36:38.260051 13241 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b14e504553b46c7bdfc8a5ae828477f" candidate_uuid: "eaeef5241eb24a8eaa0b92e919c96b8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "462d914ce8f14ee9af2b267aa392f1b4" is_pre_election: true
I20260812 02:36:38.260047 13083 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b14e504553b46c7bdfc8a5ae828477f" candidate_uuid: "eaeef5241eb24a8eaa0b92e919c96b8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2001fa695c0411996e682951e2f7307" is_pre_election: true
I20260812 02:36:38.260540 13241 raft_consensus.cc:2454] T 7b14e504553b46c7bdfc8a5ae828477f P 462d914ce8f14ee9af2b267aa392f1b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eaeef5241eb24a8eaa0b92e919c96b8b in term 0.
I20260812 02:36:38.260540 13083 raft_consensus.cc:2454] T 7b14e504553b46c7bdfc8a5ae828477f P e2001fa695c0411996e682951e2f7307 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eaeef5241eb24a8eaa0b92e919c96b8b in term 0.
I20260812 02:36:38.261191 12845 leader_election.cc:302] T 7b14e504553b46c7bdfc8a5ae828477f P eaeef5241eb24a8eaa0b92e919c96b8b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 462d914ce8f14ee9af2b267aa392f1b4, eaeef5241eb24a8eaa0b92e919c96b8b; no voters:
I20260812 02:36:38.261615 13329 raft_consensus.cc:2781] T 7b14e504553b46c7bdfc8a5ae828477f P eaeef5241eb24a8eaa0b92e919c96b8b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 02:36:38.261772 13329 raft_consensus.cc:482] T 7b14e504553b46c7bdfc8a5ae828477f P eaeef5241eb24a8eaa0b92e919c96b8b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 02:36:38.261901 13329 raft_consensus.cc:3037] T 7b14e504553b46c7bdfc8a5ae828477f P eaeef5241eb24a8eaa0b92e919c96b8b [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:36:38.264113 13329 raft_consensus.cc:504] T 7b14e504553b46c7bdfc8a5ae828477f P eaeef5241eb24a8eaa0b92e919c96b8b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2001fa695c0411996e682951e2f7307" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37419 } } peers { permanent_uuid: "eaeef5241eb24a8eaa0b92e919c96b8b" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 39939 } } peers { permanent_uuid: "462d914ce8f14ee9af2b267aa392f1b4" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 39419 } }
I20260812 02:36:38.264678 13329 leader_election.cc:288] T 7b14e504553b46c7bdfc8a5ae828477f P eaeef5241eb24a8eaa0b92e919c96b8b [CANDIDATE]: Term 1 election: Requested vote from peers e2001fa695c0411996e682951e2f7307 (127.7.132.130:37419), 462d914ce8f14ee9af2b267aa392f1b4 (127.7.132.131:39419)
I20260812 02:36:38.265169 13083 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b14e504553b46c7bdfc8a5ae828477f" candidate_uuid: "eaeef5241eb24a8eaa0b92e919c96b8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2001fa695c0411996e682951e2f7307"
I20260812 02:36:38.265169 13241 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b14e504553b46c7bdfc8a5ae828477f" candidate_uuid: "eaeef5241eb24a8eaa0b92e919c96b8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "462d914ce8f14ee9af2b267aa392f1b4"
I20260812 02:36:38.265383 13241 raft_consensus.cc:3037] T 7b14e504553b46c7bdfc8a5ae828477f P 462d914ce8f14ee9af2b267aa392f1b4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:36:38.265383 13083 raft_consensus.cc:3037] T 7b14e504553b46c7bdfc8a5ae828477f P e2001fa695c0411996e682951e2f7307 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:36:38.268687 13083 raft_consensus.cc:2454] T 7b14e504553b46c7bdfc8a5ae828477f P e2001fa695c0411996e682951e2f7307 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eaeef5241eb24a8eaa0b92e919c96b8b in term 1.
I20260812 02:36:38.268685 13241 raft_consensus.cc:2454] T 7b14e504553b46c7bdfc8a5ae828477f P 462d914ce8f14ee9af2b267aa392f1b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eaeef5241eb24a8eaa0b92e919c96b8b in term 1.
I20260812 02:36:38.269155 12847 leader_election.cc:302] T 7b14e504553b46c7bdfc8a5ae828477f P eaeef5241eb24a8eaa0b92e919c96b8b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e2001fa695c0411996e682951e2f7307, eaeef5241eb24a8eaa0b92e919c96b8b; no voters:
I20260812 02:36:38.269474 13329 raft_consensus.cc:2781] T 7b14e504553b46c7bdfc8a5ae828477f P eaeef5241eb24a8eaa0b92e919c96b8b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:36:38.269899 13329 raft_consensus.cc:686] T 7b14e504553b46c7bdfc8a5ae828477f P eaeef5241eb24a8eaa0b92e919c96b8b [term 1 LEADER]: Becoming Leader. State: Replica: eaeef5241eb24a8eaa0b92e919c96b8b, State: Running, Role: LEADER
I20260812 02:36:38.270419 13329 consensus_queue.cc:237] T 7b14e504553b46c7bdfc8a5ae828477f P eaeef5241eb24a8eaa0b92e919c96b8b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2001fa695c0411996e682951e2f7307" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37419 } } peers { permanent_uuid: "eaeef5241eb24a8eaa0b92e919c96b8b" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 39939 } } peers { permanent_uuid: "462d914ce8f14ee9af2b267aa392f1b4" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 39419 } }
I20260812 02:36:38.274972 12750 catalog_manager.cc:5495] T 7b14e504553b46c7bdfc8a5ae828477f P eaeef5241eb24a8eaa0b92e919c96b8b reported cstate change: term changed from 0 to 1, leader changed from <none> to eaeef5241eb24a8eaa0b92e919c96b8b (127.7.132.129). New cstate: current_term: 1 leader_uuid: "eaeef5241eb24a8eaa0b92e919c96b8b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2001fa695c0411996e682951e2f7307" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eaeef5241eb24a8eaa0b92e919c96b8b" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "462d914ce8f14ee9af2b267aa392f1b4" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 39419 } health_report { overall_health: UNKNOWN } } }
I20260812 02:36:38.317740 13318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:38.312783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37636 (local address 127.7.132.129:39939)
0812 02:36:38.312963 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:38.312967 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:38.313015 (+ 48us) server_negotiation.cc:407] Connection header received
0812 02:36:38.313103 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:38.313107 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:38.313195 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:38.313327 (+ 132us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 02:36:38.313668 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:38.314500 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:38.315797 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:38.316235 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:38.316328 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:38.316748 (+ 420us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 02:36:38.317262 (+ 514us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 02:36:38.317375 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:38.317478 (+ 103us) server_negotiation.cc:299] Negotiation successful
0812 02:36:38.317543 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 02:36:38.326371 13083 raft_consensus.cc:1261] T 7b14e504553b46c7bdfc8a5ae828477f P e2001fa695c0411996e682951e2f7307 [term 1 FOLLOWER]: Refusing update from remote peer eaeef5241eb24a8eaa0b92e919c96b8b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:36:38.326372 13241 raft_consensus.cc:1261] T 7b14e504553b46c7bdfc8a5ae828477f P 462d914ce8f14ee9af2b267aa392f1b4 [term 1 FOLLOWER]: Refusing update from remote peer eaeef5241eb24a8eaa0b92e919c96b8b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:36:38.327580 13332 consensus_queue.cc:1035] T 7b14e504553b46c7bdfc8a5ae828477f P eaeef5241eb24a8eaa0b92e919c96b8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "462d914ce8f14ee9af2b267aa392f1b4" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 39419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:36:38.327996 13329 consensus_queue.cc:1035] T 7b14e504553b46c7bdfc8a5ae828477f P eaeef5241eb24a8eaa0b92e919c96b8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2001fa695c0411996e682951e2f7307" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:36:38.358713 12748 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52642:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 02:36:38.359009 12748 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52642:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 02:36:38.362739 12748 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b69b5c82025242658e1cedf42a6b3440: Sending DeleteTablet for 3 replicas of tablet 7b14e504553b46c7bdfc8a5ae828477f
I20260812 02:36:38.363971 13058 tablet_service.cc:1514] Processing DeleteTablet for tablet 7b14e504553b46c7bdfc8a5ae828477f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:36:38 UTC) from {username='kudu', principal='kudu/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:34070
I20260812 02:36:38.364190 12900 tablet_service.cc:1514] Processing DeleteTablet for tablet 7b14e504553b46c7bdfc8a5ae828477f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:36:38 UTC) from {username='kudu', principal='kudu/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:37634
I20260812 02:36:38.364320 13218 tablet_service.cc:1514] Processing DeleteTablet for tablet 7b14e504553b46c7bdfc8a5ae828477f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:36:38 UTC) from {username='kudu', principal='kudu/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:46856
I20260812 02:36:38.367324 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 12822
I20260812 02:36:38.376556 13347 tablet_replica.cc:330] T 7b14e504553b46c7bdfc8a5ae828477f P e2001fa695c0411996e682951e2f7307: stopping tablet replica
I20260812 02:36:38.377245 13347 raft_consensus.cc:2229] T 7b14e504553b46c7bdfc8a5ae828477f P e2001fa695c0411996e682951e2f7307 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 02:36:38.378046 13347 raft_consensus.cc:2258] T 7b14e504553b46c7bdfc8a5ae828477f P e2001fa695c0411996e682951e2f7307 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:36:38.380582 13347 ts_tablet_manager.cc:1905] T 7b14e504553b46c7bdfc8a5ae828477f P e2001fa695c0411996e682951e2f7307: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:36:38.389815 13347 ts_tablet_manager.cc:1918] T 7b14e504553b46c7bdfc8a5ae828477f P e2001fa695c0411996e682951e2f7307: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 02:36:38.390236 13347 log.cc:1196] T 7b14e504553b46c7bdfc8a5ae828477f P e2001fa695c0411996e682951e2f7307: Deleting WAL directory at /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502156971023-7698-0/minicluster-data/ts-1/wal/wals/7b14e504553b46c7bdfc8a5ae828477f
I20260812 02:36:38.391018 13347 ts_tablet_manager.cc:1939] T 7b14e504553b46c7bdfc8a5ae828477f P e2001fa695c0411996e682951e2f7307: Deleting consensus metadata
I20260812 02:36:38.394920 12734 catalog_manager.cc:4841] TS e2001fa695c0411996e682951e2f7307 (127.7.132.130:37419): tablet 7b14e504553b46c7bdfc8a5ae828477f (table test-table [id=4e12075f15194bf689bdbf30440a88d9]) successfully deleted
I20260812 02:36:38.396778 13350 tablet_replica.cc:330] T 7b14e504553b46c7bdfc8a5ae828477f P 462d914ce8f14ee9af2b267aa392f1b4: stopping tablet replica
I20260812 02:36:38.397482 13350 raft_consensus.cc:2229] T 7b14e504553b46c7bdfc8a5ae828477f P 462d914ce8f14ee9af2b267aa392f1b4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 02:36:38.398764 13349 tablet_replica.cc:330] T 7b14e504553b46c7bdfc8a5ae828477f P eaeef5241eb24a8eaa0b92e919c96b8b: stopping tablet replica
I20260812 02:36:38.398862 13350 raft_consensus.cc:2258] T 7b14e504553b46c7bdfc8a5ae828477f P 462d914ce8f14ee9af2b267aa392f1b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:36:38.399581 13349 raft_consensus.cc:2229] T 7b14e504553b46c7bdfc8a5ae828477f P eaeef5241eb24a8eaa0b92e919c96b8b [term 1 LEADER]: Raft consensus shutting down.
I20260812 02:36:38.400445 13349 raft_consensus.cc:2258] T 7b14e504553b46c7bdfc8a5ae828477f P eaeef5241eb24a8eaa0b92e919c96b8b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:36:38.401160 13350 ts_tablet_manager.cc:1905] T 7b14e504553b46c7bdfc8a5ae828477f P 462d914ce8f14ee9af2b267aa392f1b4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:36:38.403771 13349 ts_tablet_manager.cc:1905] T 7b14e504553b46c7bdfc8a5ae828477f P eaeef5241eb24a8eaa0b92e919c96b8b: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 12 02:36:38 dist-test-slave-mn4r krb5kdc[12699](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502195, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.132.129@KRBTEST.COM
I20260812 02:36:38.408424 13350 ts_tablet_manager.cc:1918] T 7b14e504553b46c7bdfc8a5ae828477f P 462d914ce8f14ee9af2b267aa392f1b4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 02:36:38.408717 13350 log.cc:1196] T 7b14e504553b46c7bdfc8a5ae828477f P 462d914ce8f14ee9af2b267aa392f1b4: Deleting WAL directory at /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502156971023-7698-0/minicluster-data/ts-2/wal/wals/7b14e504553b46c7bdfc8a5ae828477f
I20260812 02:36:38.409307 13350 ts_tablet_manager.cc:1939] T 7b14e504553b46c7bdfc8a5ae828477f P 462d914ce8f14ee9af2b267aa392f1b4: Deleting consensus metadata
I20260812 02:36:38.411391 12731 catalog_manager.cc:4841] TS 462d914ce8f14ee9af2b267aa392f1b4 (127.7.132.131:39419): tablet 7b14e504553b46c7bdfc8a5ae828477f (table test-table [id=4e12075f15194bf689bdbf30440a88d9]) successfully deleted
I20260812 02:36:38.412842 13349 ts_tablet_manager.cc:1918] T 7b14e504553b46c7bdfc8a5ae828477f P eaeef5241eb24a8eaa0b92e919c96b8b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 02:36:38.413048 13349 log.cc:1196] T 7b14e504553b46c7bdfc8a5ae828477f P eaeef5241eb24a8eaa0b92e919c96b8b: Deleting WAL directory at /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786502156971023-7698-0/minicluster-data/ts-0/wal/wals/7b14e504553b46c7bdfc8a5ae828477f
I20260812 02:36:38.413594 13349 ts_tablet_manager.cc:1939] T 7b14e504553b46c7bdfc8a5ae828477f P eaeef5241eb24a8eaa0b92e919c96b8b: Deleting consensus metadata
I20260812 02:36:38.414798 13318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:38.369257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37642 (local address 127.7.132.129:39939)
0812 02:36:38.369493 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:38.369503 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:38.393898 (+ 24395us) server_negotiation.cc:407] Connection header received
0812 02:36:38.394517 (+ 619us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:38.394523 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:38.394621 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:38.394698 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:38.396319 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:38.397696 (+ 1377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:38.399639 (+ 1943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:38.400315 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:38.400432 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:38.406204 (+ 5772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:38.406218 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:38.406236 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:38.406294 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:38.410812 (+ 4518us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:38.411827 (+ 1015us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:38.411836 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:38.411840 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:38.412023 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:38.412521 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:38.412526 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:38.412530 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:38.413039 (+ 509us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:38.413367 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:38.414008 (+ 641us) server_negotiation.cc:299] Negotiation successful
0812 02:36:38.414340 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260812 02:36:38.415675 12972 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:36:38.415774 12734 catalog_manager.cc:4841] TS eaeef5241eb24a8eaa0b92e919c96b8b (127.7.132.129:39939): tablet 7b14e504553b46c7bdfc8a5ae828477f (table test-table [id=4e12075f15194bf689bdbf30440a88d9]) successfully deleted
I20260812 02:36:38.518770 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 12822
I20260812 02:36:38.542579 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 12989
Aug 12 02:36:38 dist-test-slave-mn4r krb5kdc[12699](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502195, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.132.130@KRBTEST.COM
I20260812 02:36:38.557356 13314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:38.543326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34084 (local address 127.7.132.130:37419)
0812 02:36:38.543529 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:38.543533 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:38.543552 (+ 19us) server_negotiation.cc:407] Connection header received
0812 02:36:38.543707 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:38.543711 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:38.543786 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:38.543868 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:38.545042 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:38.545765 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:38.547018 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:38.547388 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:38.547480 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:38.551500 (+ 4020us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:38.551514 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:38.551529 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:38.551562 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:38.554545 (+ 2983us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:38.555242 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:38.555249 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:38.555252 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:38.555404 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:38.555788 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:38.555794 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:38.555797 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:38.556155 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:38.556461 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:38.556777 (+ 316us) server_negotiation.cc:299] Negotiation successful
0812 02:36:38.557003 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260812 02:36:38.558013 13133 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:36:38.648114 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 12989
I20260812 02:36:38.669612 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 13144
Aug 12 02:36:38 dist-test-slave-mn4r krb5kdc[12699](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502195, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.132.131@KRBTEST.COM
I20260812 02:36:38.684974 13316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:38.670511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46862 (local address 127.7.132.131:39419)
0812 02:36:38.670724 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:38.670732 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:38.670753 (+ 21us) server_negotiation.cc:407] Connection header received
0812 02:36:38.670963 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:38.670971 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:38.671058 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:38.671169 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:38.672488 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:38.673217 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:38.674575 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:38.675018 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:38.675166 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:38.679601 (+ 4435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:38.679617 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:38.679639 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:38.679680 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:38.682363 (+ 2683us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:38.683053 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:38.683060 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:38.683062 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:38.683205 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:38.683546 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:38.683549 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:38.683551 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:38.684016 (+ 465us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:38.684273 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:38.684521 (+ 248us) server_negotiation.cc:299] Negotiation successful
0812 02:36:38.684716 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 02:36:38.685670 13291 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:36:38.775664 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 13144
I20260812 02:36:38.799078 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 12714
I20260812 02:36:38.800279 12786 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:36:38.910028 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 12714
2026-08-12T02:36:38Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (4780 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-12T02:36:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T02:36:38Z Disabled control of system clock
I20260812 02:36:38.990480 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502156971023-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502156971023-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502156971023-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:38351
--webserver_interface=127.7.132.190
--webserver_port=0
--builtin_ntp_servers=127.7.132.148:46881
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.132.190:38351
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260812 02:36:39.328053 13361 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:36:39.328361 13361 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:36:39.328533 13361 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:36:39.335831 13361 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:36:39.335942 13361 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:36:39.335986 13361 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:36:39.336076 13361 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:36:39.336143 13361 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:36:39.347193 13361 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:46881
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502156971023-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502156971023-7698-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.132.190:38351
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:38351
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.132.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502156971023-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:36:39.348999 13361 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:36:39.351655 13361 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:36:39.363979 13370 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:36:39.363857 13368 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:36:39.364459 13367 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:36:39.365458 13361 server_base.cc:1034] running on GCE node
I20260812 02:36:39.366343 13361 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:36:39.367784 13361 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:36:39.369051 13361 hybrid_clock.cc:648] HybridClock initialized: now 1786502199369011 us; error 32 us; skew 500 ppm
I20260812 02:36:39.369419 13361 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 02:36:39.371841 13361 webserver.cc:460] Webserver started at http://127.7.132.190:35027/ using document root <none> and password file <none>
I20260812 02:36:39.372566 13361 fs_manager.cc:359] Metadata directory not provided
I20260812 02:36:39.372702 13361 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:36:39.373056 13361 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:36:39.376760 13361 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502156971023-7698-0/minicluster-data/master-0/data/instance:
uuid: "8ec69aa619894c919b22db4063511e05"
format_stamp: "Formatted at 2026-08-12 02:36:39 on dist-test-slave-mn4r"
server_key: "1293813e187aab4aa966b72e432f62e4"
server_key_iv: "d44956f83abeb228d6d6ed3f8965398f"
server_key_version: "encryptionkey@0"
I20260812 02:36:39.377630 13361 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502156971023-7698-0/minicluster-data/master-0/wal/instance:
uuid: "8ec69aa619894c919b22db4063511e05"
format_stamp: "Formatted at 2026-08-12 02:36:39 on dist-test-slave-mn4r"
server_key: "1293813e187aab4aa966b72e432f62e4"
server_key_iv: "d44956f83abeb228d6d6ed3f8965398f"
server_key_version: "encryptionkey@0"
I20260812 02:36:39.383024 13361 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.003s
I20260812 02:36:39.386548 13378 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:36:39.387923 13361 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260812 02:36:39.388149 13361 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502156971023-7698-0/minicluster-data/master-0/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502156971023-7698-0/minicluster-data/master-0/wal
uuid: "8ec69aa619894c919b22db4063511e05"
format_stamp: "Formatted at 2026-08-12 02:36:39 on dist-test-slave-mn4r"
server_key: "1293813e187aab4aa966b72e432f62e4"
server_key_iv: "d44956f83abeb228d6d6ed3f8965398f"
server_key_version: "encryptionkey@0"
I20260812 02:36:39.388374 13361 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502156971023-7698-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502156971023-7698-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502156971023-7698-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:36:39.394342 13361 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:36:39.395133 13361 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:36:39.395452 13361 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:36:39.414192 13361 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.190:38351
I20260812 02:36:39.414196 13436 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.190:38351 every 8 connection(s)
I20260812 02:36:39.415603 13361 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
I20260812 02:36:39.419534 13437 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:36:39.421864 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 13361
I20260812 02:36:39.422132 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502156971023-7698-0/minicluster-data/master-0/wal/instance
I20260812 02:36:39.422986 7698 external_mini_cluster.cc:1428] Setting key 38b9ab1432508160834c9d04690548ce
I20260812 02:36:39.432318 13441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:39.425330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53448 (local address 127.7.132.190:38351)
0812 02:36:39.426459 (+ 1129us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:39.426470 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:39.426516 (+ 46us) server_negotiation.cc:407] Connection header received
0812 02:36:39.428226 (+ 1710us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:39.428251 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:39.428706 (+ 455us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:39.429135 (+ 429us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:39.430428 (+ 1293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:39.430434 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:39.430458 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:36:39.430473 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:39.430537 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:36:39.430552 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:39.430834 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:39.430976 (+ 142us) server_negotiation.cc:299] Negotiation successful
0812 02:36:39.431347 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":613,"thread_start_us":375,"threads_started":1}
I20260812 02:36:39.432644 13437 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ec69aa619894c919b22db4063511e05: Bootstrap starting.
I20260812 02:36:39.436431 13437 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8ec69aa619894c919b22db4063511e05: Neither blocks nor log segments found. Creating new log.
I20260812 02:36:39.437469 13437 log.cc:824] T 00000000000000000000000000000000 P 8ec69aa619894c919b22db4063511e05: Log is configured to *not* fsync() on all Append() calls
I20260812 02:36:39.440290 13437 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ec69aa619894c919b22db4063511e05: No bootstrap required, opened a new log
I20260812 02:36:39.448271 13437 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8ec69aa619894c919b22db4063511e05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ec69aa619894c919b22db4063511e05" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 38351 } }
I20260812 02:36:39.448668 13437 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8ec69aa619894c919b22db4063511e05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:36:39.448777 13437 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8ec69aa619894c919b22db4063511e05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ec69aa619894c919b22db4063511e05, State: Initialized, Role: FOLLOWER
I20260812 02:36:39.449342 13437 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8ec69aa619894c919b22db4063511e05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ec69aa619894c919b22db4063511e05" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 38351 } }
I20260812 02:36:39.449594 13437 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8ec69aa619894c919b22db4063511e05 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 02:36:39.449750 13437 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8ec69aa619894c919b22db4063511e05 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 02:36:39.449919 13437 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8ec69aa619894c919b22db4063511e05 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:36:39.452086 13437 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8ec69aa619894c919b22db4063511e05 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ec69aa619894c919b22db4063511e05" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 38351 } }
I20260812 02:36:39.452616 13437 leader_election.cc:302] T 00000000000000000000000000000000 P 8ec69aa619894c919b22db4063511e05 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8ec69aa619894c919b22db4063511e05; no voters:
I20260812 02:36:39.453226 13437 leader_election.cc:288] T 00000000000000000000000000000000 P 8ec69aa619894c919b22db4063511e05 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 02:36:39.453373 13443 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8ec69aa619894c919b22db4063511e05 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:36:39.454185 13443 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8ec69aa619894c919b22db4063511e05 [term 1 LEADER]: Becoming Leader. State: Replica: 8ec69aa619894c919b22db4063511e05, State: Running, Role: LEADER
I20260812 02:36:39.454836 13443 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8ec69aa619894c919b22db4063511e05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ec69aa619894c919b22db4063511e05" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 38351 } }
I20260812 02:36:39.455677 13437 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8ec69aa619894c919b22db4063511e05 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:36:39.457690 13445 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ec69aa619894c919b22db4063511e05 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8ec69aa619894c919b22db4063511e05. Latest consensus state: current_term: 1 leader_uuid: "8ec69aa619894c919b22db4063511e05" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ec69aa619894c919b22db4063511e05" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 38351 } } }
I20260812 02:36:39.458014 13445 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ec69aa619894c919b22db4063511e05 [sys.catalog]: This master's current role is: LEADER
I20260812 02:36:39.458384 13444 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ec69aa619894c919b22db4063511e05 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8ec69aa619894c919b22db4063511e05" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ec69aa619894c919b22db4063511e05" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 38351 } } }
I20260812 02:36:39.462656 13444 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ec69aa619894c919b22db4063511e05 [sys.catalog]: This master's current role is: LEADER
I20260812 02:36:39.463377 13453 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:36:39.469305 13453 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 02:36:39.477807 13453 catalog_manager.cc:1347] Generated new cluster ID: 278259c9746344659bfcfce60f1e462f
I20260812 02:36:39.477927 13453 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 02:36:39.518347 13453 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 02:36:39.518872 13453 catalog_manager.cc:1504] Loading token signing keys...
I20260812 02:36:39.529501 13453 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8ec69aa619894c919b22db4063511e05: Generated new TSK 0
I20260812 02:36:39.530444 13453 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 02:36:39.535240 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502156971023-7698-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502156971023-7698-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502156971023-7698-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502156971023-7698-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.129:0
--local_ip_for_outbound_sockets=127.7.132.129
--webserver_interface=127.7.132.129
--webserver_port=0
--tserver_master_addrs=127.7.132.190:38351
--builtin_ntp_servers=127.7.132.148:46881
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260812 02:36:39.883679 13469 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:36:39.883948 13469 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:36:39.884078 13469 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:36:39.891182 13469 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:36:39.891278 13469 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:36:39.891387 13469 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.129
I20260812 02:36:39.900763 13469 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:46881
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502156971023-7698-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502156971023-7698-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502156971023-7698-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.132.129
--webserver_port=0
--tserver_master_addrs=127.7.132.190:38351
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.129
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502156971023-7698-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:36:39.902041 13469 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:36:39.903821 13469 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:36:39.917232 13475 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:36:39.917310 13479 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:36:39.917207 13476 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:36:39.918013 13469 server_base.cc:1034] running on GCE node
I20260812 02:36:39.918782 13469 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:36:39.919765 13469 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:36:39.920965 13469 hybrid_clock.cc:648] HybridClock initialized: now 1786502199920925 us; error 32 us; skew 500 ppm
I20260812 02:36:39.921309 13469 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 02:36:39.923566 13469 webserver.cc:460] Webserver started at http://127.7.132.129:37753/ using document root <none> and password file <none>
I20260812 02:36:39.924296 13469 fs_manager.cc:359] Metadata directory not provided
I20260812 02:36:39.924412 13469 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:36:39.924748 13469 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:36:39.927693 13469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502156971023-7698-0/minicluster-data/ts-0/data/instance:
uuid: "5231b46700ec42dda91422aa4b394bb4"
format_stamp: "Formatted at 2026-08-12 02:36:39 on dist-test-slave-mn4r"
server_key: "0c73cd7df36ae671e398fd9934504c59"
server_key_iv: "98cf779d673f8ab3e72f1c90c165eb06"
server_key_version: "encryptionkey@0"
I20260812 02:36:39.928515 13469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502156971023-7698-0/minicluster-data/ts-0/wal/instance:
uuid: "5231b46700ec42dda91422aa4b394bb4"
format_stamp: "Formatted at 2026-08-12 02:36:39 on dist-test-slave-mn4r"
server_key: "0c73cd7df36ae671e398fd9934504c59"
server_key_iv: "98cf779d673f8ab3e72f1c90c165eb06"
server_key_version: "encryptionkey@0"
I20260812 02:36:39.934244 13469 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260812 02:36:39.937597 13486 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:36:39.939086 13469 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 02:36:39.939271 13469 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502156971023-7698-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502156971023-7698-0/minicluster-data/ts-0/wal
uuid: "5231b46700ec42dda91422aa4b394bb4"
format_stamp: "Formatted at 2026-08-12 02:36:39 on dist-test-slave-mn4r"
server_key: "0c73cd7df36ae671e398fd9934504c59"
server_key_iv: "98cf779d673f8ab3e72f1c90c165eb06"
server_key_version: "encryptionkey@0"
I20260812 02:36:39.939505 13469 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502156971023-7698-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502156971023-7698-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502156971023-7698-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:36:39.945107 13469 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:36:39.945899 13469 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:36:39.946231 13469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:36:39.947121 13469 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:36:39.948457 13469 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:36:39.948580 13469 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:36:39.948710 13469 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:36:39.948798 13469 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:36:39.982477 13469 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.129:36285
I20260812 02:36:39.982535 13621 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.129:36285 every 8 connection(s)
I20260812 02:36:39.983852 13469 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502156971023-7698-0/minicluster-data/ts-0/data/info.pb
I20260812 02:36:39.992511 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 13469
I20260812 02:36:39.992730 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502156971023-7698-0/minicluster-data/ts-0/wal/instance
I20260812 02:36:39.992800 13624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:39.986598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:57797 (local address 127.7.132.190:38351)
0812 02:36:39.987029 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:39.987034 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:39.988346 (+ 1312us) server_negotiation.cc:407] Connection header received
0812 02:36:39.990069 (+ 1723us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:39.990075 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:39.990185 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:39.990308 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:39.991566 (+ 1258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:39.991572 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:39.991577 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:36:39.991597 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:39.991647 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:36:39.991659 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:39.991794 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:39.992433 (+ 639us) server_negotiation.cc:299] Negotiation successful
0812 02:36:39.992498 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":324,"thread_start_us":209,"threads_started":1}
I20260812 02:36:39.993445 13625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:39.987257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:38351 (local address 127.7.132.129:57797)
0812 02:36:39.988174 (+ 917us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:39.988219 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:39.989796 (+ 1577us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:39.990538 (+ 742us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:39.990546 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:39.990642 (+ 96us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:36:39.991277 (+ 635us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 02:36:39.991280 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:36:39.991316 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 02:36:39.991398 (+ 82us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:36:39.991759 (+ 361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:36:39.991773 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:36:39.991781 (+ 8us) client_negotiation.cc:769] Sending connection context
0812 02:36:39.992086 (+ 305us) client_negotiation.cc:240] Negotiation successful
0812 02:36:39.992346 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":281,"threads_started":1}
I20260812 02:36:39.993487 7698 external_mini_cluster.cc:1428] Setting key 2659e757d940cc5bc9b2d7b31e7a6673
I20260812 02:36:39.995349 13622 heartbeater.cc:347] Connected to a master server at 127.7.132.190:38351
I20260812 02:36:39.995689 13622 heartbeater.cc:464] Registering TS with master...
I20260812 02:36:39.996291 13622 heartbeater.cc:511] Master 127.7.132.190:38351 requested a full tablet report, sending...
I20260812 02:36:39.996858 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502156971023-7698-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502156971023-7698-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502156971023-7698-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502156971023-7698-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.130:0
--local_ip_for_outbound_sockets=127.7.132.130
--webserver_interface=127.7.132.130
--webserver_port=0
--tserver_master_addrs=127.7.132.190:38351
--builtin_ntp_servers=127.7.132.148:46881
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260812 02:36:39.998661 13396 ts_manager.cc:194] Registered new tserver with Master: 5231b46700ec42dda91422aa4b394bb4 (127.7.132.129:36285)
W20260812 02:36:40.333341 13626 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:36:40.333623 13626 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:36:40.333714 13626 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:36:40.341022 13626 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:36:40.341117 13626 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:36:40.341226 13626 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.130
I20260812 02:36:40.350538 13626 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:46881
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502156971023-7698-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502156971023-7698-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502156971023-7698-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.132.130
--webserver_port=0
--tserver_master_addrs=127.7.132.190:38351
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.130
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502156971023-7698-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:36:40.351833 13626 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:36:40.353642 13626 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:36:40.367273 13634 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:36:40.367375 13636 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:36:40.367379 13633 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:36:40.368314 13626 server_base.cc:1034] running on GCE node
I20260812 02:36:40.368947 13626 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:36:40.369969 13626 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:36:40.371300 13626 hybrid_clock.cc:648] HybridClock initialized: now 1786502200371238 us; error 53 us; skew 500 ppm
I20260812 02:36:40.371596 13626 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 02:36:40.373898 13626 webserver.cc:460] Webserver started at http://127.7.132.130:39903/ using document root <none> and password file <none>
I20260812 02:36:40.374655 13626 fs_manager.cc:359] Metadata directory not provided
I20260812 02:36:40.374758 13626 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:36:40.375036 13626 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:36:40.377879 13626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502156971023-7698-0/minicluster-data/ts-1/data/instance:
uuid: "c527e23d9e954f10b513004163d7b4bb"
format_stamp: "Formatted at 2026-08-12 02:36:40 on dist-test-slave-mn4r"
server_key: "5fbdb80cee58bf764177e44f3d605fcb"
server_key_iv: "6f2f9d79d5d80f369081b7e70ea05189"
server_key_version: "encryptionkey@0"
I20260812 02:36:40.378706 13626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502156971023-7698-0/minicluster-data/ts-1/wal/instance:
uuid: "c527e23d9e954f10b513004163d7b4bb"
format_stamp: "Formatted at 2026-08-12 02:36:40 on dist-test-slave-mn4r"
server_key: "5fbdb80cee58bf764177e44f3d605fcb"
server_key_iv: "6f2f9d79d5d80f369081b7e70ea05189"
server_key_version: "encryptionkey@0"
I20260812 02:36:40.383867 13626 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 02:36:40.387284 13644 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:36:40.388628 13626 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260812 02:36:40.388800 13626 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502156971023-7698-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502156971023-7698-0/minicluster-data/ts-1/wal
uuid: "c527e23d9e954f10b513004163d7b4bb"
format_stamp: "Formatted at 2026-08-12 02:36:40 on dist-test-slave-mn4r"
server_key: "5fbdb80cee58bf764177e44f3d605fcb"
server_key_iv: "6f2f9d79d5d80f369081b7e70ea05189"
server_key_version: "encryptionkey@0"
I20260812 02:36:40.389039 13626 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502156971023-7698-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502156971023-7698-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502156971023-7698-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:36:40.395099 13626 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:36:40.395905 13626 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:36:40.396191 13626 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:36:40.397189 13626 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:36:40.399042 13626 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:36:40.399173 13626 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:36:40.399312 13626 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:36:40.399399 13626 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:36:40.436445 13626 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.130:45193
I20260812 02:36:40.436501 13790 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.130:45193 every 8 connection(s)
I20260812 02:36:40.437801 13626 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502156971023-7698-0/minicluster-data/ts-1/data/info.pb
I20260812 02:36:40.441605 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 13626
I20260812 02:36:40.441788 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502156971023-7698-0/minicluster-data/ts-1/wal/instance
I20260812 02:36:40.442355 7698 external_mini_cluster.cc:1428] Setting key 75979226c472955c6b5dce65174a75e1
I20260812 02:36:40.445631 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502156971023-7698-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502156971023-7698-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502156971023-7698-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502156971023-7698-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.131:0
--local_ip_for_outbound_sockets=127.7.132.131
--webserver_interface=127.7.132.131
--webserver_port=0
--tserver_master_addrs=127.7.132.190:38351
--builtin_ntp_servers=127.7.132.148:46881
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260812 02:36:40.446866 13624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:40.440489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:37511 (local address 127.7.132.190:38351)
0812 02:36:40.440682 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:40.440686 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:40.442350 (+ 1664us) server_negotiation.cc:407] Connection header received
0812 02:36:40.444481 (+ 2131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:40.444486 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:40.444544 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:40.444644 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:40.446206 (+ 1562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:40.446211 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:40.446214 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:36:40.446228 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:40.446259 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:36:40.446270 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:40.446361 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:40.446620 (+ 259us) server_negotiation.cc:299] Negotiation successful
0812 02:36:40.446670 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260812 02:36:40.448256 13793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:40.441041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:38351 (local address 127.7.132.130:37511)
0812 02:36:40.442166 (+ 1125us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:40.442217 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:40.444185 (+ 1968us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:40.444922 (+ 737us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:40.444932 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:40.445063 (+ 131us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:36:40.445789 (+ 726us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 02:36:40.445794 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:36:40.445854 (+ 60us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 02:36:40.445967 (+ 113us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:36:40.446373 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:36:40.446387 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:36:40.446398 (+ 11us) client_negotiation.cc:769] Sending connection context
0812 02:36:40.446695 (+ 297us) client_negotiation.cc:240] Negotiation successful
0812 02:36:40.446961 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":539,"thread_start_us":350,"threads_started":1}
I20260812 02:36:40.449572 13791 heartbeater.cc:347] Connected to a master server at 127.7.132.190:38351
I20260812 02:36:40.449887 13791 heartbeater.cc:464] Registering TS with master...
I20260812 02:36:40.450558 13791 heartbeater.cc:511] Master 127.7.132.190:38351 requested a full tablet report, sending...
I20260812 02:36:40.451910 13396 ts_manager.cc:194] Registered new tserver with Master: c527e23d9e954f10b513004163d7b4bb (127.7.132.130:45193)
W20260812 02:36:40.783420 13794 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:36:40.783700 13794 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:36:40.783788 13794 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:36:40.791093 13794 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:36:40.791193 13794 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:36:40.791303 13794 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.131
I20260812 02:36:40.800709 13794 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:46881
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502156971023-7698-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502156971023-7698-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502156971023-7698-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.132.131
--webserver_port=0
--tserver_master_addrs=127.7.132.190:38351
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.131
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502156971023-7698-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:36:40.801975 13794 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:36:40.803887 13794 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:36:40.817584 13800 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:36:40.817641 13799 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:36:40.817874 13802 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:36:40.818439 13794 server_base.cc:1034] running on GCE node
I20260812 02:36:40.819149 13794 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:36:40.820182 13794 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:36:40.821466 13794 hybrid_clock.cc:648] HybridClock initialized: now 1786502200821383 us; error 62 us; skew 500 ppm
I20260812 02:36:40.821832 13794 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 02:36:40.824247 13794 webserver.cc:460] Webserver started at http://127.7.132.131:44615/ using document root <none> and password file <none>
I20260812 02:36:40.824987 13794 fs_manager.cc:359] Metadata directory not provided
I20260812 02:36:40.825125 13794 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:36:40.825475 13794 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:36:40.828665 13794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502156971023-7698-0/minicluster-data/ts-2/data/instance:
uuid: "c5cdbe58ec674e9e8d3af99bb1dc7451"
format_stamp: "Formatted at 2026-08-12 02:36:40 on dist-test-slave-mn4r"
server_key: "caac7973188b2e1d116d85cd9f44f720"
server_key_iv: "1c0f36732b7c138dba28da7bc3738e15"
server_key_version: "encryptionkey@0"
I20260812 02:36:40.829492 13794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502156971023-7698-0/minicluster-data/ts-2/wal/instance:
uuid: "c5cdbe58ec674e9e8d3af99bb1dc7451"
format_stamp: "Formatted at 2026-08-12 02:36:40 on dist-test-slave-mn4r"
server_key: "caac7973188b2e1d116d85cd9f44f720"
server_key_iv: "1c0f36732b7c138dba28da7bc3738e15"
server_key_version: "encryptionkey@0"
I20260812 02:36:40.834951 13794 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.008s
I20260812 02:36:40.838393 13812 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:36:40.839744 13794 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260812 02:36:40.839916 13794 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502156971023-7698-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502156971023-7698-0/minicluster-data/ts-2/wal
uuid: "c5cdbe58ec674e9e8d3af99bb1dc7451"
format_stamp: "Formatted at 2026-08-12 02:36:40 on dist-test-slave-mn4r"
server_key: "caac7973188b2e1d116d85cd9f44f720"
server_key_iv: "1c0f36732b7c138dba28da7bc3738e15"
server_key_version: "encryptionkey@0"
I20260812 02:36:40.840175 13794 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502156971023-7698-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502156971023-7698-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502156971023-7698-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:36:40.846352 13794 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:36:40.847165 13794 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:36:40.847442 13794 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:36:40.848392 13794 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:36:40.849807 13794 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:36:40.849936 13794 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:36:40.850070 13794 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:36:40.850175 13794 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260812 02:36:40.883414 13794 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.131:42235
I20260812 02:36:40.883453 13944 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.131:42235 every 8 connection(s)
I20260812 02:36:40.884764 13794 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502156971023-7698-0/minicluster-data/ts-2/data/info.pb
I20260812 02:36:40.889678 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 13794
I20260812 02:36:40.889859 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502156971023-7698-0/minicluster-data/ts-2/wal/instance
I20260812 02:36:40.890378 7698 external_mini_cluster.cc:1428] Setting key e086535932a104373b47afe7b56edd0a
I20260812 02:36:40.893982 13624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:40.887806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:48791 (local address 127.7.132.190:38351)
0812 02:36:40.887996 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:40.888001 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:40.889443 (+ 1442us) server_negotiation.cc:407] Connection header received
0812 02:36:40.891432 (+ 1989us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:40.891438 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:40.891513 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:40.891644 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:40.893228 (+ 1584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:40.893235 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:40.893244 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:36:40.893260 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:40.893291 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:36:40.893302 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:40.893430 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:40.893634 (+ 204us) server_negotiation.cc:299] Negotiation successful
0812 02:36:40.893705 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260812 02:36:40.895053 13947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:40.888367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:38351 (local address 127.7.132.131:48791)
0812 02:36:40.889289 (+ 922us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:40.889326 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:40.891117 (+ 1791us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:40.891930 (+ 813us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:40.891940 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:40.892077 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:36:40.892898 (+ 821us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 02:36:40.892902 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:36:40.892961 (+ 59us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 02:36:40.893051 (+ 90us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:36:40.893401 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:36:40.893412 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:36:40.893420 (+ 8us) client_negotiation.cc:769] Sending connection context
0812 02:36:40.893726 (+ 306us) client_negotiation.cc:240] Negotiation successful
0812 02:36:40.894118 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":274,"threads_started":1}
I20260812 02:36:40.896337 13945 heartbeater.cc:347] Connected to a master server at 127.7.132.190:38351
I20260812 02:36:40.896663 13945 heartbeater.cc:464] Registering TS with master...
I20260812 02:36:40.897254 13945 heartbeater.cc:511] Master 127.7.132.190:38351 requested a full tablet report, sending...
I20260812 02:36:40.898644 13396 ts_manager.cc:194] Registered new tserver with Master: c5cdbe58ec674e9e8d3af99bb1dc7451 (127.7.132.131:42235)
I20260812 02:36:40.906318 7698 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 02:36:40.913589 13624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:40.911114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53462 (local address 127.7.132.190:38351)
0812 02:36:40.911291 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:40.911296 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:40.911796 (+ 500us) server_negotiation.cc:407] Connection header received
0812 02:36:40.911955 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:40.911959 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:40.912017 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:40.912113 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:40.913088 (+ 975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:40.913098 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:40.913105 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:36:40.913133 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:40.913166 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:36:40.913179 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:40.913287 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:40.913323 (+ 36us) server_negotiation.cc:299] Negotiation successful
0812 02:36:40.913369 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 02:36:40.917833 13396 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53462:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 02:36:40.924700 13396 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 02:36:40.945576 13962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:40.937959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.130:45193 (local address 127.0.0.1:36360)
0812 02:36:40.940173 (+ 2214us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:40.940195 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:40.940303 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:40.942517 (+ 2214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:40.942527 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:40.942651 (+ 124us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:36:40.943634 (+ 983us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 02:36:40.943640 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:36:40.943689 (+ 49us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 02:36:40.943832 (+ 143us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:36:40.944829 (+ 997us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:36:40.944841 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:36:40.944848 (+ 7us) client_negotiation.cc:769] Sending connection context
0812 02:36:40.945031 (+ 183us) client_negotiation.cc:240] Negotiation successful
0812 02:36:40.945151 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2046,"thread_start_us":954,"threads_started":1}
I20260812 02:36:40.946106 13964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:40.937492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.131:42235 (local address 127.0.0.1:42048)
0812 02:36:40.939665 (+ 2173us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:40.939707 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:40.939843 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:40.943489 (+ 3646us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:40.943494 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:40.943527 (+ 33us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:36:40.944531 (+ 1004us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 02:36:40.944533 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:36:40.944668 (+ 135us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 02:36:40.944726 (+ 58us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:36:40.945395 (+ 669us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:36:40.945399 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:36:40.945405 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 02:36:40.945588 (+ 183us) client_negotiation.cc:240] Negotiation successful
0812 02:36:40.945657 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1900,"thread_start_us":1691,"threads_started":1}
I20260812 02:36:40.946118 13963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:40.937676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42048 (local address 127.7.132.131:42235)
0812 02:36:40.938881 (+ 1205us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:40.938893 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:40.940018 (+ 1125us) server_negotiation.cc:407] Connection header received
0812 02:36:40.941412 (+ 1394us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:40.941430 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:40.941719 (+ 289us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:40.941923 (+ 204us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:40.944906 (+ 2983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:40.945003 (+ 97us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:40.945017 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:36:40.945038 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:40.945134 (+ 96us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:36:40.945161 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:40.945333 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:40.945594 (+ 261us) server_negotiation.cc:299] Negotiation successful
0812 02:36:40.945706 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":997,"thread_start_us":822,"threads_started":1}
I20260812 02:36:40.946985 13965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:40.938593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36360 (local address 127.7.132.130:45193)
0812 02:36:40.939712 (+ 1119us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:40.939720 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:40.940453 (+ 733us) server_negotiation.cc:407] Connection header received
0812 02:36:40.940553 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:40.940565 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:40.940867 (+ 302us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:40.941059 (+ 192us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:40.944006 (+ 2947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:40.944015 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:40.944032 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:36:40.944068 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:40.944136 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:36:40.944163 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:40.944355 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:40.946442 (+ 2087us) server_negotiation.cc:299] Negotiation successful
0812 02:36:40.946526 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":944,"thread_start_us":986,"threads_started":1}
I20260812 02:36:40.948104 13968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:40.939816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:36285 (local address 127.0.0.1:47402)
0812 02:36:40.944570 (+ 4754us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:40.944586 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:40.944695 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:40.946217 (+ 1522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:40.946225 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:40.946256 (+ 31us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:36:40.946894 (+ 638us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 02:36:40.946896 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:36:40.946908 (+ 12us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 02:36:40.946941 (+ 33us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:36:40.947470 (+ 529us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:36:40.947474 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:36:40.947477 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 02:36:40.947636 (+ 159us) client_negotiation.cc:240] Negotiation successful
0812 02:36:40.947686 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4320,"thread_start_us":4204,"threads_started":1}
I20260812 02:36:40.948103 13969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:40.940158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47402 (local address 127.7.132.129:36285)
0812 02:36:40.944528 (+ 4370us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:40.944536 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:40.944670 (+ 134us) server_negotiation.cc:407] Connection header received
0812 02:36:40.944817 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:40.944827 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:40.945043 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:40.945239 (+ 196us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:40.947143 (+ 1904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:40.947154 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:40.947170 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:36:40.947203 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:40.947266 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:36:40.947285 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:40.947414 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:40.947630 (+ 216us) server_negotiation.cc:299] Negotiation successful
0812 02:36:40.947725 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4209,"thread_start_us":3138,"threads_started":1}
I20260812 02:36:40.950937 13869 tablet_service.cc:1467] Processing CreateTablet for tablet 1693dbeb70e0427bb7af434768b1e676 (DEFAULT_TABLE table=test-table [id=428eafc7a56d47afad359de1f5568a4c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:36:40.950934 13703 tablet_service.cc:1467] Processing CreateTablet for tablet 1693dbeb70e0427bb7af434768b1e676 (DEFAULT_TABLE table=test-table [id=428eafc7a56d47afad359de1f5568a4c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:36:40.951985 13540 tablet_service.cc:1467] Processing CreateTablet for tablet 1693dbeb70e0427bb7af434768b1e676 (DEFAULT_TABLE table=test-table [id=428eafc7a56d47afad359de1f5568a4c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:36:40.952463 13703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1693dbeb70e0427bb7af434768b1e676. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:36:40.952458 13869 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1693dbeb70e0427bb7af434768b1e676. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:36:40.953109 13540 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1693dbeb70e0427bb7af434768b1e676. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:36:40.964726 13970 tablet_bootstrap.cc:492] T 1693dbeb70e0427bb7af434768b1e676 P 5231b46700ec42dda91422aa4b394bb4: Bootstrap starting.
I20260812 02:36:40.968406 13970 tablet_bootstrap.cc:654] T 1693dbeb70e0427bb7af434768b1e676 P 5231b46700ec42dda91422aa4b394bb4: Neither blocks nor log segments found. Creating new log.
I20260812 02:36:40.969409 13970 log.cc:824] T 1693dbeb70e0427bb7af434768b1e676 P 5231b46700ec42dda91422aa4b394bb4: Log is configured to *not* fsync() on all Append() calls
I20260812 02:36:40.972249 13970 tablet_bootstrap.cc:492] T 1693dbeb70e0427bb7af434768b1e676 P 5231b46700ec42dda91422aa4b394bb4: No bootstrap required, opened a new log
I20260812 02:36:40.972519 13970 ts_tablet_manager.cc:1397] T 1693dbeb70e0427bb7af434768b1e676 P 5231b46700ec42dda91422aa4b394bb4: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260812 02:36:40.974377 13972 tablet_bootstrap.cc:492] T 1693dbeb70e0427bb7af434768b1e676 P c527e23d9e954f10b513004163d7b4bb: Bootstrap starting.
I20260812 02:36:40.976197 13971 tablet_bootstrap.cc:492] T 1693dbeb70e0427bb7af434768b1e676 P c5cdbe58ec674e9e8d3af99bb1dc7451: Bootstrap starting.
I20260812 02:36:40.977617 13972 tablet_bootstrap.cc:654] T 1693dbeb70e0427bb7af434768b1e676 P c527e23d9e954f10b513004163d7b4bb: Neither blocks nor log segments found. Creating new log.
I20260812 02:36:40.978724 13972 log.cc:824] T 1693dbeb70e0427bb7af434768b1e676 P c527e23d9e954f10b513004163d7b4bb: Log is configured to *not* fsync() on all Append() calls
I20260812 02:36:40.979869 13971 tablet_bootstrap.cc:654] T 1693dbeb70e0427bb7af434768b1e676 P c5cdbe58ec674e9e8d3af99bb1dc7451: Neither blocks nor log segments found. Creating new log.
I20260812 02:36:40.981173 13971 log.cc:824] T 1693dbeb70e0427bb7af434768b1e676 P c5cdbe58ec674e9e8d3af99bb1dc7451: Log is configured to *not* fsync() on all Append() calls
I20260812 02:36:40.981418 13972 tablet_bootstrap.cc:492] T 1693dbeb70e0427bb7af434768b1e676 P c527e23d9e954f10b513004163d7b4bb: No bootstrap required, opened a new log
I20260812 02:36:40.982406 13972 ts_tablet_manager.cc:1397] T 1693dbeb70e0427bb7af434768b1e676 P c527e23d9e954f10b513004163d7b4bb: Time spent bootstrapping tablet: real 0.012s user 0.007s sys 0.000s
I20260812 02:36:40.982980 13970 raft_consensus.cc:348] T 1693dbeb70e0427bb7af434768b1e676 P 5231b46700ec42dda91422aa4b394bb4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5cdbe58ec674e9e8d3af99bb1dc7451" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 42235 } } peers { permanent_uuid: "5231b46700ec42dda91422aa4b394bb4" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 36285 } } peers { permanent_uuid: "c527e23d9e954f10b513004163d7b4bb" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 45193 } }
I20260812 02:36:40.983595 13970 raft_consensus.cc:374] T 1693dbeb70e0427bb7af434768b1e676 P 5231b46700ec42dda91422aa4b394bb4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:36:40.983773 13970 raft_consensus.cc:729] T 1693dbeb70e0427bb7af434768b1e676 P 5231b46700ec42dda91422aa4b394bb4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5231b46700ec42dda91422aa4b394bb4, State: Initialized, Role: FOLLOWER
I20260812 02:36:40.984368 13971 tablet_bootstrap.cc:492] T 1693dbeb70e0427bb7af434768b1e676 P c5cdbe58ec674e9e8d3af99bb1dc7451: No bootstrap required, opened a new log
I20260812 02:36:40.984688 13971 ts_tablet_manager.cc:1397] T 1693dbeb70e0427bb7af434768b1e676 P c5cdbe58ec674e9e8d3af99bb1dc7451: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.008s
I20260812 02:36:40.984558 13970 consensus_queue.cc:260] T 1693dbeb70e0427bb7af434768b1e676 P 5231b46700ec42dda91422aa4b394bb4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5cdbe58ec674e9e8d3af99bb1dc7451" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 42235 } } peers { permanent_uuid: "5231b46700ec42dda91422aa4b394bb4" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 36285 } } peers { permanent_uuid: "c527e23d9e954f10b513004163d7b4bb" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 45193 } }
I20260812 02:36:40.996701 13972 raft_consensus.cc:348] T 1693dbeb70e0427bb7af434768b1e676 P c527e23d9e954f10b513004163d7b4bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5cdbe58ec674e9e8d3af99bb1dc7451" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 42235 } } peers { permanent_uuid: "5231b46700ec42dda91422aa4b394bb4" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 36285 } } peers { permanent_uuid: "c527e23d9e954f10b513004163d7b4bb" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 45193 } }
I20260812 02:36:40.997020 13971 raft_consensus.cc:348] T 1693dbeb70e0427bb7af434768b1e676 P c5cdbe58ec674e9e8d3af99bb1dc7451 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5cdbe58ec674e9e8d3af99bb1dc7451" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 42235 } } peers { permanent_uuid: "5231b46700ec42dda91422aa4b394bb4" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 36285 } } peers { permanent_uuid: "c527e23d9e954f10b513004163d7b4bb" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 45193 } }
I20260812 02:36:40.997502 13971 raft_consensus.cc:374] T 1693dbeb70e0427bb7af434768b1e676 P c5cdbe58ec674e9e8d3af99bb1dc7451 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:36:40.997519 13972 raft_consensus.cc:374] T 1693dbeb70e0427bb7af434768b1e676 P c527e23d9e954f10b513004163d7b4bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:36:40.997643 13971 raft_consensus.cc:729] T 1693dbeb70e0427bb7af434768b1e676 P c5cdbe58ec674e9e8d3af99bb1dc7451 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5cdbe58ec674e9e8d3af99bb1dc7451, State: Initialized, Role: FOLLOWER
I20260812 02:36:40.997756 13972 raft_consensus.cc:729] T 1693dbeb70e0427bb7af434768b1e676 P c527e23d9e954f10b513004163d7b4bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c527e23d9e954f10b513004163d7b4bb, State: Initialized, Role: FOLLOWER
I20260812 02:36:40.998719 13972 consensus_queue.cc:260] T 1693dbeb70e0427bb7af434768b1e676 P c527e23d9e954f10b513004163d7b4bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5cdbe58ec674e9e8d3af99bb1dc7451" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 42235 } } peers { permanent_uuid: "5231b46700ec42dda91422aa4b394bb4" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 36285 } } peers { permanent_uuid: "c527e23d9e954f10b513004163d7b4bb" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 45193 } }
I20260812 02:36:40.999290 13622 heartbeater.cc:503] Master 127.7.132.190:38351 was elected leader, sending a full tablet report...
I20260812 02:36:40.998392 13971 consensus_queue.cc:260] T 1693dbeb70e0427bb7af434768b1e676 P c5cdbe58ec674e9e8d3af99bb1dc7451 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5cdbe58ec674e9e8d3af99bb1dc7451" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 42235 } } peers { permanent_uuid: "5231b46700ec42dda91422aa4b394bb4" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 36285 } } peers { permanent_uuid: "c527e23d9e954f10b513004163d7b4bb" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 45193 } }
I20260812 02:36:41.000058 13970 ts_tablet_manager.cc:1428] T 1693dbeb70e0427bb7af434768b1e676 P 5231b46700ec42dda91422aa4b394bb4: Time spent starting tablet: real 0.027s user 0.020s sys 0.007s
I20260812 02:36:41.001121 13945 heartbeater.cc:503] Master 127.7.132.190:38351 was elected leader, sending a full tablet report...
I20260812 02:36:41.001748 13971 ts_tablet_manager.cc:1428] T 1693dbeb70e0427bb7af434768b1e676 P c5cdbe58ec674e9e8d3af99bb1dc7451: Time spent starting tablet: real 0.017s user 0.013s sys 0.000s
I20260812 02:36:41.003713 13791 heartbeater.cc:503] Master 127.7.132.190:38351 was elected leader, sending a full tablet report...
I20260812 02:36:41.005079 13972 ts_tablet_manager.cc:1428] T 1693dbeb70e0427bb7af434768b1e676 P c527e23d9e954f10b513004163d7b4bb: Time spent starting tablet: real 0.022s user 0.016s sys 0.004s
I20260812 02:36:41.040629 13978 raft_consensus.cc:482] T 1693dbeb70e0427bb7af434768b1e676 P c5cdbe58ec674e9e8d3af99bb1dc7451 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 02:36:41.040931 13978 raft_consensus.cc:504] T 1693dbeb70e0427bb7af434768b1e676 P c5cdbe58ec674e9e8d3af99bb1dc7451 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5cdbe58ec674e9e8d3af99bb1dc7451" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 42235 } } peers { permanent_uuid: "5231b46700ec42dda91422aa4b394bb4" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 36285 } } peers { permanent_uuid: "c527e23d9e954f10b513004163d7b4bb" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 45193 } }
I20260812 02:36:41.042825 13978 leader_election.cc:288] T 1693dbeb70e0427bb7af434768b1e676 P c5cdbe58ec674e9e8d3af99bb1dc7451 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5231b46700ec42dda91422aa4b394bb4 (127.7.132.129:36285), c527e23d9e954f10b513004163d7b4bb (127.7.132.130:45193)
I20260812 02:36:41.045902 13947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:41.043209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.130:45193 (local address 127.7.132.131:38961)
0812 02:36:41.043568 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:41.043595 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:41.043765 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:41.044234 (+ 469us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:41.044238 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:41.044259 (+ 21us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:36:41.044953 (+ 694us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 02:36:41.044957 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:36:41.044982 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 02:36:41.045041 (+ 59us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:36:41.045447 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:36:41.045451 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:36:41.045454 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 02:36:41.045599 (+ 145us) client_negotiation.cc:240] Negotiation successful
0812 02:36:41.045661 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":227}
I20260812 02:36:41.045966 13965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:41.043359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:38961 (local address 127.7.132.130:45193)
0812 02:36:41.043564 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:41.043570 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:41.043706 (+ 136us) server_negotiation.cc:407] Connection header received
0812 02:36:41.043936 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:41.043943 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:41.044017 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:41.044142 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:41.045252 (+ 1110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:41.045258 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:41.045264 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:36:41.045285 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:41.045324 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:36:41.045343 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:41.045470 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:41.045591 (+ 121us) server_negotiation.cc:299] Negotiation successful
0812 02:36:41.045641 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260812 02:36:41.046501 13969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:41.043328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:48637 (local address 127.7.132.129:36285)
0812 02:36:41.043534 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:41.043541 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:41.044292 (+ 751us) server_negotiation.cc:407] Connection header received
0812 02:36:41.044511 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:41.044519 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:41.044586 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:41.044707 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:41.045787 (+ 1080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:41.045793 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:41.045798 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:36:41.045817 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:41.045851 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:36:41.045866 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:41.046001 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:41.046214 (+ 213us) server_negotiation.cc:299] Negotiation successful
0812 02:36:41.046272 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 02:36:41.046728 13980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:41.043259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:36285 (local address 127.7.132.131:48637)
0812 02:36:41.044199 (+ 940us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:41.044216 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:41.044329 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:41.044762 (+ 433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:41.044774 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:41.044792 (+ 18us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:36:41.045586 (+ 794us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 02:36:41.045589 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:36:41.045605 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 02:36:41.045649 (+ 44us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:36:41.045988 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:36:41.045993 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:36:41.045999 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 02:36:41.046192 (+ 193us) client_negotiation.cc:240] Negotiation successful
0812 02:36:41.046247 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":816,"thread_start_us":527,"threads_started":1}
I20260812 02:36:41.047197 13728 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1693dbeb70e0427bb7af434768b1e676" candidate_uuid: "c5cdbe58ec674e9e8d3af99bb1dc7451" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c527e23d9e954f10b513004163d7b4bb" is_pre_election: true
I20260812 02:36:41.047712 13728 raft_consensus.cc:2454] T 1693dbeb70e0427bb7af434768b1e676 P c527e23d9e954f10b513004163d7b4bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c5cdbe58ec674e9e8d3af99bb1dc7451 in term 0.
I20260812 02:36:41.047742 13563 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1693dbeb70e0427bb7af434768b1e676" candidate_uuid: "c5cdbe58ec674e9e8d3af99bb1dc7451" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5231b46700ec42dda91422aa4b394bb4" is_pre_election: true
I20260812 02:36:41.048321 13563 raft_consensus.cc:2454] T 1693dbeb70e0427bb7af434768b1e676 P 5231b46700ec42dda91422aa4b394bb4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c5cdbe58ec674e9e8d3af99bb1dc7451 in term 0.
I20260812 02:36:41.048509 13814 leader_election.cc:302] T 1693dbeb70e0427bb7af434768b1e676 P c5cdbe58ec674e9e8d3af99bb1dc7451 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c527e23d9e954f10b513004163d7b4bb, c5cdbe58ec674e9e8d3af99bb1dc7451; no voters:
I20260812 02:36:41.048950 13978 raft_consensus.cc:2781] T 1693dbeb70e0427bb7af434768b1e676 P c5cdbe58ec674e9e8d3af99bb1dc7451 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 02:36:41.049109 13978 raft_consensus.cc:482] T 1693dbeb70e0427bb7af434768b1e676 P c5cdbe58ec674e9e8d3af99bb1dc7451 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 02:36:41.049238 13978 raft_consensus.cc:3037] T 1693dbeb70e0427bb7af434768b1e676 P c5cdbe58ec674e9e8d3af99bb1dc7451 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:36:41.051596 13978 raft_consensus.cc:504] T 1693dbeb70e0427bb7af434768b1e676 P c5cdbe58ec674e9e8d3af99bb1dc7451 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5cdbe58ec674e9e8d3af99bb1dc7451" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 42235 } } peers { permanent_uuid: "5231b46700ec42dda91422aa4b394bb4" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 36285 } } peers { permanent_uuid: "c527e23d9e954f10b513004163d7b4bb" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 45193 } }
I20260812 02:36:41.052192 13978 leader_election.cc:288] T 1693dbeb70e0427bb7af434768b1e676 P c5cdbe58ec674e9e8d3af99bb1dc7451 [CANDIDATE]: Term 1 election: Requested vote from peers 5231b46700ec42dda91422aa4b394bb4 (127.7.132.129:36285), c527e23d9e954f10b513004163d7b4bb (127.7.132.130:45193)
I20260812 02:36:41.052791 13728 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1693dbeb70e0427bb7af434768b1e676" candidate_uuid: "c5cdbe58ec674e9e8d3af99bb1dc7451" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c527e23d9e954f10b513004163d7b4bb"
I20260812 02:36:41.052870 13563 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1693dbeb70e0427bb7af434768b1e676" candidate_uuid: "c5cdbe58ec674e9e8d3af99bb1dc7451" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5231b46700ec42dda91422aa4b394bb4"
I20260812 02:36:41.053014 13728 raft_consensus.cc:3037] T 1693dbeb70e0427bb7af434768b1e676 P c527e23d9e954f10b513004163d7b4bb [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:36:41.053128 13563 raft_consensus.cc:3037] T 1693dbeb70e0427bb7af434768b1e676 P 5231b46700ec42dda91422aa4b394bb4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:36:41.055544 13563 raft_consensus.cc:2454] T 1693dbeb70e0427bb7af434768b1e676 P 5231b46700ec42dda91422aa4b394bb4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c5cdbe58ec674e9e8d3af99bb1dc7451 in term 1.
I20260812 02:36:41.055830 13728 raft_consensus.cc:2454] T 1693dbeb70e0427bb7af434768b1e676 P c527e23d9e954f10b513004163d7b4bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c5cdbe58ec674e9e8d3af99bb1dc7451 in term 1.
I20260812 02:36:41.056010 13813 leader_election.cc:302] T 1693dbeb70e0427bb7af434768b1e676 P c5cdbe58ec674e9e8d3af99bb1dc7451 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5231b46700ec42dda91422aa4b394bb4, c5cdbe58ec674e9e8d3af99bb1dc7451; no voters:
I20260812 02:36:41.056370 13978 raft_consensus.cc:2781] T 1693dbeb70e0427bb7af434768b1e676 P c5cdbe58ec674e9e8d3af99bb1dc7451 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:36:41.056879 13978 raft_consensus.cc:686] T 1693dbeb70e0427bb7af434768b1e676 P c5cdbe58ec674e9e8d3af99bb1dc7451 [term 1 LEADER]: Becoming Leader. State: Replica: c5cdbe58ec674e9e8d3af99bb1dc7451, State: Running, Role: LEADER
I20260812 02:36:41.057390 13978 consensus_queue.cc:237] T 1693dbeb70e0427bb7af434768b1e676 P c5cdbe58ec674e9e8d3af99bb1dc7451 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5cdbe58ec674e9e8d3af99bb1dc7451" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 42235 } } peers { permanent_uuid: "5231b46700ec42dda91422aa4b394bb4" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 36285 } } peers { permanent_uuid: "c527e23d9e954f10b513004163d7b4bb" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 45193 } }
I20260812 02:36:41.062886 13395 catalog_manager.cc:5495] T 1693dbeb70e0427bb7af434768b1e676 P c5cdbe58ec674e9e8d3af99bb1dc7451 reported cstate change: term changed from 0 to 1, leader changed from <none> to c5cdbe58ec674e9e8d3af99bb1dc7451 (127.7.132.131). New cstate: current_term: 1 leader_uuid: "c5cdbe58ec674e9e8d3af99bb1dc7451" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5cdbe58ec674e9e8d3af99bb1dc7451" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 42235 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5231b46700ec42dda91422aa4b394bb4" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 36285 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c527e23d9e954f10b513004163d7b4bb" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 45193 } health_report { overall_health: UNKNOWN } } }
I20260812 02:36:41.099794 13963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:41.097614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42056 (local address 127.7.132.131:42235)
0812 02:36:41.097801 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:41.097806 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:41.097825 (+ 19us) server_negotiation.cc:407] Connection header received
0812 02:36:41.098002 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:41.098006 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:41.098110 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:41.098205 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:41.099173 (+ 968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:41.099177 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:41.099181 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:36:41.099194 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:41.099224 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:36:41.099235 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:41.099336 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:41.099432 (+ 96us) server_negotiation.cc:299] Negotiation successful
0812 02:36:41.099480 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260812 02:36:41.108529 13728 raft_consensus.cc:1261] T 1693dbeb70e0427bb7af434768b1e676 P c527e23d9e954f10b513004163d7b4bb [term 1 FOLLOWER]: Refusing update from remote peer c5cdbe58ec674e9e8d3af99bb1dc7451: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:36:41.108556 13563 raft_consensus.cc:1261] T 1693dbeb70e0427bb7af434768b1e676 P 5231b46700ec42dda91422aa4b394bb4 [term 1 FOLLOWER]: Refusing update from remote peer c5cdbe58ec674e9e8d3af99bb1dc7451: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:36:41.109647 13978 consensus_queue.cc:1035] T 1693dbeb70e0427bb7af434768b1e676 P c5cdbe58ec674e9e8d3af99bb1dc7451 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5231b46700ec42dda91422aa4b394bb4" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 36285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:36:41.109932 13981 consensus_queue.cc:1035] T 1693dbeb70e0427bb7af434768b1e676 P c5cdbe58ec674e9e8d3af99bb1dc7451 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c527e23d9e954f10b513004163d7b4bb" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 45193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:36:41.135097 13396 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:53462:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 02:36:41.135412 13396 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:53462:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 02:36:41.139209 13396 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8ec69aa619894c919b22db4063511e05: Sending DeleteTablet for 3 replicas of tablet 1693dbeb70e0427bb7af434768b1e676
I20260812 02:36:41.140283 13869 tablet_service.cc:1514] Processing DeleteTablet for tablet 1693dbeb70e0427bb7af434768b1e676 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:36:41 UTC) from {username='slave'} at 127.0.0.1:42048
I20260812 02:36:41.140439 13703 tablet_service.cc:1514] Processing DeleteTablet for tablet 1693dbeb70e0427bb7af434768b1e676 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:36:41 UTC) from {username='slave'} at 127.0.0.1:36360
I20260812 02:36:41.141067 13540 tablet_service.cc:1514] Processing DeleteTablet for tablet 1693dbeb70e0427bb7af434768b1e676 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:36:41 UTC) from {username='slave'} at 127.0.0.1:47402
I20260812 02:36:41.143493 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 13469
I20260812 02:36:41.156500 13996 tablet_replica.cc:330] T 1693dbeb70e0427bb7af434768b1e676 P c527e23d9e954f10b513004163d7b4bb: stopping tablet replica
I20260812 02:36:41.157282 13996 raft_consensus.cc:2229] T 1693dbeb70e0427bb7af434768b1e676 P c527e23d9e954f10b513004163d7b4bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 02:36:41.158432 13996 raft_consensus.cc:2258] T 1693dbeb70e0427bb7af434768b1e676 P c527e23d9e954f10b513004163d7b4bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:36:41.160418 13969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:41.145371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47418 (local address 127.7.132.129:36285)
0812 02:36:41.145605 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:41.145614 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:41.148934 (+ 3320us) server_negotiation.cc:407] Connection header received
0812 02:36:41.149134 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:41.149142 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:41.149279 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:41.149397 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:41.159123 (+ 9726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:41.159132 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:41.159138 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:36:41.159167 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:41.159217 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:36:41.159238 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:41.159377 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:41.159619 (+ 242us) server_negotiation.cc:299] Negotiation successful
0812 02:36:41.159693 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260812 02:36:41.159036 13997 tablet_replica.cc:330] T 1693dbeb70e0427bb7af434768b1e676 P 5231b46700ec42dda91422aa4b394bb4: stopping tablet replica
I20260812 02:36:41.160933 13996 ts_tablet_manager.cc:1905] T 1693dbeb70e0427bb7af434768b1e676 P c527e23d9e954f10b513004163d7b4bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:36:41.160979 13995 tablet_replica.cc:330] T 1693dbeb70e0427bb7af434768b1e676 P c5cdbe58ec674e9e8d3af99bb1dc7451: stopping tablet replica
I20260812 02:36:41.161218 13614 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:36:41.161563 13995 raft_consensus.cc:2229] T 1693dbeb70e0427bb7af434768b1e676 P c5cdbe58ec674e9e8d3af99bb1dc7451 [term 1 LEADER]: Raft consensus shutting down.
I20260812 02:36:41.162549 13995 raft_consensus.cc:2258] T 1693dbeb70e0427bb7af434768b1e676 P c5cdbe58ec674e9e8d3af99bb1dc7451 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:36:41.165333 13995 ts_tablet_manager.cc:1905] T 1693dbeb70e0427bb7af434768b1e676 P c5cdbe58ec674e9e8d3af99bb1dc7451: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:36:41.168341 13996 ts_tablet_manager.cc:1918] T 1693dbeb70e0427bb7af434768b1e676 P c527e23d9e954f10b513004163d7b4bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 02:36:41.168568 13996 log.cc:1196] T 1693dbeb70e0427bb7af434768b1e676 P c527e23d9e954f10b513004163d7b4bb: Deleting WAL directory at /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502156971023-7698-0/minicluster-data/ts-1/wal/wals/1693dbeb70e0427bb7af434768b1e676
I20260812 02:36:41.169144 13996 ts_tablet_manager.cc:1939] T 1693dbeb70e0427bb7af434768b1e676 P c527e23d9e954f10b513004163d7b4bb: Deleting consensus metadata
I20260812 02:36:41.171159 13380 catalog_manager.cc:4841] TS c527e23d9e954f10b513004163d7b4bb (127.7.132.130:45193): tablet 1693dbeb70e0427bb7af434768b1e676 (table test-table [id=428eafc7a56d47afad359de1f5568a4c]) successfully deleted
I20260812 02:36:41.175487 13995 ts_tablet_manager.cc:1918] T 1693dbeb70e0427bb7af434768b1e676 P c5cdbe58ec674e9e8d3af99bb1dc7451: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 02:36:41.175683 13995 log.cc:1196] T 1693dbeb70e0427bb7af434768b1e676 P c5cdbe58ec674e9e8d3af99bb1dc7451: Deleting WAL directory at /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786502156971023-7698-0/minicluster-data/ts-2/wal/wals/1693dbeb70e0427bb7af434768b1e676
I20260812 02:36:41.176195 13995 ts_tablet_manager.cc:1939] T 1693dbeb70e0427bb7af434768b1e676 P c5cdbe58ec674e9e8d3af99bb1dc7451: Deleting consensus metadata
I20260812 02:36:41.177685 13379 catalog_manager.cc:4841] TS c5cdbe58ec674e9e8d3af99bb1dc7451 (127.7.132.131:42235): tablet 1693dbeb70e0427bb7af434768b1e676 (table test-table [id=428eafc7a56d47afad359de1f5568a4c]) successfully deleted
I20260812 02:36:41.161427 13997 raft_consensus.cc:2229] T 1693dbeb70e0427bb7af434768b1e676 P 5231b46700ec42dda91422aa4b394bb4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 02:36:41.290495 13997 raft_consensus.cc:2258] T 1693dbeb70e0427bb7af434768b1e676 P 5231b46700ec42dda91422aa4b394bb4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:36:41.290705 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 13469
W20260812 02:36:41.311655 13379 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.132.129:36285 (error 108)
I20260812 02:36:41.311971 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 13626
I20260812 02:36:41.313982 13968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:41.313447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:36285 (local address 127.0.0.1:47430)
0812 02:36:41.313668 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:41.313812 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.129:36285: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
W20260812 02:36:41.314291 13379 catalog_manager.cc:4584] TS 5231b46700ec42dda91422aa4b394bb4 (127.7.132.129:36285): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1693dbeb70e0427bb7af434768b1e676: Network error: Client connection negotiation failed: client connection to 127.7.132.129:36285: connect: Connection refused (error 111)
I20260812 02:36:41.315308 13965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:41.312964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36376 (local address 127.7.132.130:45193)
0812 02:36:41.313153 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:41.313158 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:41.313176 (+ 18us) server_negotiation.cc:407] Connection header received
0812 02:36:41.313310 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:41.313318 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:41.313438 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:41.313550 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:41.314657 (+ 1107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:41.314665 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:41.314673 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:36:41.314691 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:41.314732 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:36:41.314747 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:41.314909 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:41.314972 (+ 63us) server_negotiation.cc:299] Negotiation successful
0812 02:36:41.315041 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260812 02:36:41.316064 13779 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:36:41.368198 13968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:41.367610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:36285 (local address 127.0.0.1:47432)
0812 02:36:41.367861 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:41.367988 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.129:36285: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260812 02:36:41.369421 13968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:41.368990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:36285 (local address 127.0.0.1:47448)
0812 02:36:41.369172 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:41.369261 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.129:36285: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260812 02:36:41.409660 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 13626
I20260812 02:36:41.430157 13968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:41.429638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:36285 (local address 127.0.0.1:47452)
0812 02:36:41.429855 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:41.429942 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.129:36285: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260812 02:36:41.432569 13968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:41.432195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:36285 (local address 127.0.0.1:47458)
0812 02:36:41.432366 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:41.432453 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.129:36285: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260812 02:36:41.438992 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 13794
I20260812 02:36:41.442129 13963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:41.439822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42066 (local address 127.7.132.131:42235)
0812 02:36:41.440032 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:41.440038 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:41.440062 (+ 24us) server_negotiation.cc:407] Connection header received
0812 02:36:41.440219 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:41.440224 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:41.440275 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:41.440399 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:41.441352 (+ 953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:41.441358 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:41.441363 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:36:41.441382 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:41.441421 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:36:41.441434 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:41.441599 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:41.441677 (+ 78us) server_negotiation.cc:299] Negotiation successful
0812 02:36:41.441740 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 02:36:41.442991 13940 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:36:41.513393 13968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:41.512828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:36285 (local address 127.0.0.1:47474)
0812 02:36:41.513138 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:41.513243 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.129:36285: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147}
I20260812 02:36:41.514721 13968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:41.514268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:36285 (local address 127.0.0.1:47476)
0812 02:36:41.514517 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:41.514589 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.129:36285: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
I20260812 02:36:41.564159 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 13794
I20260812 02:36:41.587628 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 13361
I20260812 02:36:41.589031 13431 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:36:41.696847 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 13361
I20260812 02:36:41.696895 13968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:41.696209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:36285 (local address 127.0.0.1:47482)
0812 02:36:41.696525 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:41.696695 (+ 170us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.129:36285: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
2026-08-12T02:36:41Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (2787 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260812 02:36:41.746393 7698 mini_oidc.cc:152] Starting JWKS server
I20260812 02:36:41.749083 7698 webserver.cc:460] Webserver started at https://127.0.0.1:43073/ using document root <none> and password file <none>
I20260812 02:36:41.749365 7698 mini_oidc.cc:189] Starting OIDC Discovery server
I20260812 02:36:41.749953 7698 webserver.cc:460] Webserver started at http://127.0.0.1:45865/ using document root <none> and password file <none>
2026-08-12T02:36:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T02:36:41Z Disabled control of system clock
I20260812 02:36:41.782070 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786502156971023-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786502156971023-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786502156971023-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:41363
--webserver_interface=127.7.132.190
--webserver_port=0
--builtin_ntp_servers=127.7.132.148:41755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.132.190:41363
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43073/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786502156971023-7698-0/testchainca.cert with env {}
W20260812 02:36:42.114490 14014 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:36:42.114862 14014 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:36:42.115037 14014 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:36:42.122298 14014 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:36:42.122460 14014 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:36:42.122558 14014 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:36:42.122676 14014 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:36:42.122766 14014 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 02:36:42.122856 14014 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 02:36:42.122934 14014 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:43073/jwks.json
I20260812 02:36:42.132896 14014 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:41755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786502156971023-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786502156971023-7698-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.132.190:41363
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:41363
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43073/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.132.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786502156971023-7698-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786502156971023-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:36:42.134364 14014 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:36:42.135740 14014 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:36:42.148002 14020 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:36:42.147960 14022 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:36:42.147919 14024 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:36:42.149144 14014 server_base.cc:1034] running on GCE node
I20260812 02:36:42.149940 14014 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:36:42.151541 14014 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:36:42.152747 14014 hybrid_clock.cc:648] HybridClock initialized: now 1786502202152709 us; error 33 us; skew 500 ppm
I20260812 02:36:42.153115 14014 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 02:36:42.160506 14014 webserver.cc:460] Webserver started at http://127.7.132.190:43581/ using document root <none> and password file <none>
I20260812 02:36:42.161342 14014 fs_manager.cc:359] Metadata directory not provided
I20260812 02:36:42.161484 14014 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:36:42.161938 14014 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:36:42.166930 14014 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786502156971023-7698-0/minicluster-data/master-0/data/instance:
uuid: "7ce83cf5a55246cd89c82340087bcd00"
format_stamp: "Formatted at 2026-08-12 02:36:42 on dist-test-slave-mn4r"
server_key: "ececdab289cb747b11bbe4e8763b0cbe"
server_key_iv: "f6bb8f83e41b0852b1e1cb89526fb9ef"
server_key_version: "encryptionkey@0"
I20260812 02:36:42.168186 14014 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786502156971023-7698-0/minicluster-data/master-0/wal/instance:
uuid: "7ce83cf5a55246cd89c82340087bcd00"
format_stamp: "Formatted at 2026-08-12 02:36:42 on dist-test-slave-mn4r"
server_key: "ececdab289cb747b11bbe4e8763b0cbe"
server_key_iv: "f6bb8f83e41b0852b1e1cb89526fb9ef"
server_key_version: "encryptionkey@0"
I20260812 02:36:42.176066 14014 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.004s
I20260812 02:36:42.181396 14030 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:36:42.183310 14014 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260812 02:36:42.183532 14014 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786502156971023-7698-0/minicluster-data/master-0/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786502156971023-7698-0/minicluster-data/master-0/wal
uuid: "7ce83cf5a55246cd89c82340087bcd00"
format_stamp: "Formatted at 2026-08-12 02:36:42 on dist-test-slave-mn4r"
server_key: "ececdab289cb747b11bbe4e8763b0cbe"
server_key_iv: "f6bb8f83e41b0852b1e1cb89526fb9ef"
server_key_version: "encryptionkey@0"
I20260812 02:36:42.183811 14014 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786502156971023-7698-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786502156971023-7698-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786502156971023-7698-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:36:42.228070 14014 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:36:42.229148 14014 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:36:42.229501 14014 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:36:42.250710 14014 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.190:41363
I20260812 02:36:42.250730 14085 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.190:41363 every 8 connection(s)
I20260812 02:36:42.252089 14014 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
I20260812 02:36:42.256033 14086 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:36:42.259372 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 14014
I20260812 02:36:42.259624 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786502156971023-7698-0/minicluster-data/master-0/wal/instance
I20260812 02:36:42.260520 7698 external_mini_cluster.cc:1428] Setting key c6c6f098a3e15e513b91cec25c112694
I20260812 02:36:42.268718 14086 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ce83cf5a55246cd89c82340087bcd00: Bootstrap starting.
I20260812 02:36:42.272367 14086 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7ce83cf5a55246cd89c82340087bcd00: Neither blocks nor log segments found. Creating new log.
I20260812 02:36:42.273609 14086 log.cc:824] T 00000000000000000000000000000000 P 7ce83cf5a55246cd89c82340087bcd00: Log is configured to *not* fsync() on all Append() calls
I20260812 02:36:42.277541 14086 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ce83cf5a55246cd89c82340087bcd00: No bootstrap required, opened a new log
I20260812 02:36:42.278543 14090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:42.262726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34120 (local address 127.7.132.190:41363)
0812 02:36:42.263878 (+ 1152us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:42.263893 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:42.263949 (+ 56us) server_negotiation.cc:407] Connection header received
0812 02:36:42.266871 (+ 2922us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:42.266901 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:42.267517 (+ 616us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:42.267974 (+ 457us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:42.268736 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:42.270796 (+ 2060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:42.272946 (+ 2150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:42.273988 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:42.274235 (+ 247us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:42.275603 (+ 1368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:42.275613 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:42.275631 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:36:42.276314 (+ 683us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:42.276432 (+ 118us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:36:42.276459 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:42.276574 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:42.276835 (+ 261us) server_negotiation.cc:299] Negotiation successful
0812 02:36:42.277166 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":602,"thread_start_us":336,"threads_started":1}
I20260812 02:36:42.283264 7698 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 02:36:42.286805 14086 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7ce83cf5a55246cd89c82340087bcd00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ce83cf5a55246cd89c82340087bcd00" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 41363 } }
I20260812 02:36:42.287222 14086 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7ce83cf5a55246cd89c82340087bcd00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:36:42.287341 14086 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7ce83cf5a55246cd89c82340087bcd00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ce83cf5a55246cd89c82340087bcd00, State: Initialized, Role: FOLLOWER
I20260812 02:36:42.287905 14086 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7ce83cf5a55246cd89c82340087bcd00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ce83cf5a55246cd89c82340087bcd00" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 41363 } }
I20260812 02:36:42.288144 14086 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7ce83cf5a55246cd89c82340087bcd00 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 02:36:42.288287 14086 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7ce83cf5a55246cd89c82340087bcd00 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 02:36:42.288463 14086 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7ce83cf5a55246cd89c82340087bcd00 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:36:42.290819 14086 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7ce83cf5a55246cd89c82340087bcd00 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ce83cf5a55246cd89c82340087bcd00" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 41363 } }
I20260812 02:36:42.291401 14086 leader_election.cc:302] T 00000000000000000000000000000000 P 7ce83cf5a55246cd89c82340087bcd00 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7ce83cf5a55246cd89c82340087bcd00; no voters:
I20260812 02:36:42.292104 14086 leader_election.cc:288] T 00000000000000000000000000000000 P 7ce83cf5a55246cd89c82340087bcd00 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 02:36:42.292209 14093 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7ce83cf5a55246cd89c82340087bcd00 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:36:42.292943 14093 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7ce83cf5a55246cd89c82340087bcd00 [term 1 LEADER]: Becoming Leader. State: Replica: 7ce83cf5a55246cd89c82340087bcd00, State: Running, Role: LEADER
I20260812 02:36:42.293481 14093 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7ce83cf5a55246cd89c82340087bcd00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ce83cf5a55246cd89c82340087bcd00" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 41363 } }
I20260812 02:36:42.294454 14086 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7ce83cf5a55246cd89c82340087bcd00 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:36:42.296554 14095 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ce83cf5a55246cd89c82340087bcd00 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7ce83cf5a55246cd89c82340087bcd00. Latest consensus state: current_term: 1 leader_uuid: "7ce83cf5a55246cd89c82340087bcd00" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ce83cf5a55246cd89c82340087bcd00" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 41363 } } }
I20260812 02:36:42.296663 14094 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ce83cf5a55246cd89c82340087bcd00 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7ce83cf5a55246cd89c82340087bcd00" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ce83cf5a55246cd89c82340087bcd00" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 41363 } } }
I20260812 02:36:42.297097 14094 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ce83cf5a55246cd89c82340087bcd00 [sys.catalog]: This master's current role is: LEADER
I20260812 02:36:42.297137 14095 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ce83cf5a55246cd89c82340087bcd00 [sys.catalog]: This master's current role is: LEADER
I20260812 02:36:42.300069 14100 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:36:42.305478 14100 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 02:36:42.314860 14100 catalog_manager.cc:1347] Generated new cluster ID: 1e0c72ce31974a29afab8166573a47a2
I20260812 02:36:42.314996 14100 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 02:36:42.328896 14100 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 02:36:42.330271 14100 catalog_manager.cc:1504] Loading token signing keys...
I20260812 02:36:42.337353 14100 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7ce83cf5a55246cd89c82340087bcd00: Generated new TSK 0
I20260812 02:36:42.338291 14100 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260812 02:36:42.381151 14126 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260812 02:36:42.381392 14126 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260812 02:36:42.384127 14090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:42.359744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34130 (local address 127.7.132.190:41363)
0812 02:36:42.359955 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:42.359963 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:42.360517 (+ 554us) server_negotiation.cc:407] Connection header received
0812 02:36:42.360826 (+ 309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:42.360832 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:42.360941 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:42.361107 (+ 166us) server_negotiation.cc:226] Negotiated authn=JWT
0812 02:36:42.361650 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:42.362859 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:42.364591 (+ 1732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:42.365310 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:42.365435 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:42.366045 (+ 610us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 02:36:42.383275 (+ 17230us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43073/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0812 02:36:42.383811 (+ 536us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:34130: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43073/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":66}
W20260812 02:36:42.384262 14090 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:34130: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43073/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260812 02:36:42.385613 14125 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:36:42.359655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:41363 (local address 127.0.0.1:34130)
0812 02:36:42.360413 (+ 758us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:42.360430 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:42.360595 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:42.361150 (+ 555us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:42.361154 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:42.361164 (+ 10us) client_negotiation.cc:189] Negotiated authn=JWT
0812 02:36:42.361481 (+ 317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:42.361489 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:42.363055 (+ 1566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:42.363061 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:42.364431 (+ 1370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:42.364447 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:42.365516 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:42.365522 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:42.365661 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:42.365726 (+ 65us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 02:36:42.383634 (+ 17908us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43073/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0812 02:36:42.383917 (+ 283us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.7.132.190:41363: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43073/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":564,"thread_start_us":411,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.7.132.190:41363: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43073/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260812 02:36:42.387275 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 14014
I20260812 02:36:42.388509 14081 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:36:42.483147 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 14014
2026-08-12T02:36:42Z chronyd exiting
I20260812 02:36:42.538851 7698 test_util.cc:175] -----------------------------------------------
I20260812 02:36:42.538997 7698 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786502156971023-7698-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (797 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260812 02:36:42.543901 7698 mini_oidc.cc:152] Starting JWKS server
I20260812 02:36:42.545773 7698 webserver.cc:460] Webserver started at https://127.0.0.1:35603/ using document root <none> and password file <none>
I20260812 02:36:42.546025 7698 mini_oidc.cc:189] Starting OIDC Discovery server
I20260812 02:36:42.546741 7698 webserver.cc:460] Webserver started at http://127.0.0.1:43303/ using document root <none> and password file <none>
2026-08-12T02:36:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T02:36:42Z Disabled control of system clock
I20260812 02:36:42.579339 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786502156971023-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786502156971023-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786502156971023-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:42903
--webserver_interface=127.7.132.190
--webserver_port=0
--builtin_ntp_servers=127.7.132.148:39205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.132.190:42903
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35603/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786502156971023-7698-0/testchainca.cert with env {}
W20260812 02:36:42.911365 14136 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:36:42.911728 14136 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:36:42.911899 14136 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:36:42.919147 14136 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:36:42.919286 14136 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:36:42.919378 14136 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:36:42.919485 14136 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:36:42.919574 14136 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 02:36:42.919665 14136 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 02:36:42.919744 14136 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:35603/jwks.json
I20260812 02:36:42.929823 14136 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:39205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786502156971023-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786502156971023-7698-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.132.190:42903
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:42903
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35603/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.132.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786502156971023-7698-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786502156971023-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:36:42.931298 14136 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:36:42.932662 14136 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:36:42.944957 14141 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:36:42.945005 14143 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:36:42.945034 14145 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:36:42.951310 14136 server_base.cc:1034] running on GCE node
I20260812 02:36:42.952165 14136 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:36:42.953696 14136 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:36:42.954934 14136 hybrid_clock.cc:648] HybridClock initialized: now 1786502202954890 us; error 69 us; skew 500 ppm
I20260812 02:36:42.955319 14136 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 02:36:42.957856 14136 webserver.cc:460] Webserver started at http://127.7.132.190:42911/ using document root <none> and password file <none>
I20260812 02:36:42.958688 14136 fs_manager.cc:359] Metadata directory not provided
I20260812 02:36:42.958837 14136 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:36:42.959197 14136 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:36:42.962330 14136 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786502156971023-7698-0/minicluster-data/master-0/data/instance:
uuid: "046891eb17b943768c9c410f354b3093"
format_stamp: "Formatted at 2026-08-12 02:36:42 on dist-test-slave-mn4r"
server_key: "89452db39b185fe1b753deac98f2b2a9"
server_key_iv: "9da8347bfdfd9ded94b562b074d635f6"
server_key_version: "encryptionkey@0"
I20260812 02:36:42.963172 14136 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786502156971023-7698-0/minicluster-data/master-0/wal/instance:
uuid: "046891eb17b943768c9c410f354b3093"
format_stamp: "Formatted at 2026-08-12 02:36:42 on dist-test-slave-mn4r"
server_key: "89452db39b185fe1b753deac98f2b2a9"
server_key_iv: "9da8347bfdfd9ded94b562b074d635f6"
server_key_version: "encryptionkey@0"
I20260812 02:36:42.968467 14136 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 02:36:42.972016 14151 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:36:42.973301 14136 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 02:36:42.973490 14136 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786502156971023-7698-0/minicluster-data/master-0/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786502156971023-7698-0/minicluster-data/master-0/wal
uuid: "046891eb17b943768c9c410f354b3093"
format_stamp: "Formatted at 2026-08-12 02:36:42 on dist-test-slave-mn4r"
server_key: "89452db39b185fe1b753deac98f2b2a9"
server_key_iv: "9da8347bfdfd9ded94b562b074d635f6"
server_key_version: "encryptionkey@0"
I20260812 02:36:42.973721 14136 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786502156971023-7698-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786502156971023-7698-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786502156971023-7698-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:36:42.996104 14136 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:36:42.997048 14136 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:36:42.997385 14136 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:36:43.017051 14136 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.190:42903
I20260812 02:36:43.017071 14208 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.190:42903 every 8 connection(s)
I20260812 02:36:43.018617 14136 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
I20260812 02:36:43.019845 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 14136
I20260812 02:36:43.020059 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786502156971023-7698-0/minicluster-data/master-0/wal/instance
I20260812 02:36:43.020711 7698 external_mini_cluster.cc:1428] Setting key a36f0799b13275cb9d79f486b2d89883
I20260812 02:36:43.023298 14209 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:36:43.036906 14212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:43.022775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46902 (local address 127.7.132.190:42903)
0812 02:36:43.023895 (+ 1120us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:43.023910 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:43.023959 (+ 49us) server_negotiation.cc:407] Connection header received
0812 02:36:43.026279 (+ 2320us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:43.026307 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:43.026942 (+ 635us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:43.027407 (+ 465us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:43.027953 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:43.030095 (+ 2142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:43.031350 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:43.032401 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:43.032593 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:43.033859 (+ 1266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:43.033870 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:43.033890 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:36:43.034692 (+ 802us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:43.034788 (+ 96us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:36:43.034823 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:43.034954 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:43.035156 (+ 202us) server_negotiation.cc:299] Negotiation successful
0812 02:36:43.035545 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":594,"thread_start_us":352,"threads_started":1}
I20260812 02:36:43.037989 14209 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 046891eb17b943768c9c410f354b3093: Bootstrap starting.
I20260812 02:36:43.041229 7698 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 02:36:43.041934 14209 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 046891eb17b943768c9c410f354b3093: Neither blocks nor log segments found. Creating new log.
I20260812 02:36:43.043272 14209 log.cc:824] T 00000000000000000000000000000000 P 046891eb17b943768c9c410f354b3093: Log is configured to *not* fsync() on all Append() calls
I20260812 02:36:43.046454 14209 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 046891eb17b943768c9c410f354b3093: No bootstrap required, opened a new log
I20260812 02:36:43.054116 14209 raft_consensus.cc:348] T 00000000000000000000000000000000 P 046891eb17b943768c9c410f354b3093 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "046891eb17b943768c9c410f354b3093" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 42903 } }
I20260812 02:36:43.054471 14209 raft_consensus.cc:374] T 00000000000000000000000000000000 P 046891eb17b943768c9c410f354b3093 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:36:43.054549 14209 raft_consensus.cc:729] T 00000000000000000000000000000000 P 046891eb17b943768c9c410f354b3093 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 046891eb17b943768c9c410f354b3093, State: Initialized, Role: FOLLOWER
I20260812 02:36:43.055054 14209 consensus_queue.cc:260] T 00000000000000000000000000000000 P 046891eb17b943768c9c410f354b3093 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "046891eb17b943768c9c410f354b3093" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 42903 } }
I20260812 02:36:43.055248 14209 raft_consensus.cc:388] T 00000000000000000000000000000000 P 046891eb17b943768c9c410f354b3093 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 02:36:43.055346 14209 raft_consensus.cc:482] T 00000000000000000000000000000000 P 046891eb17b943768c9c410f354b3093 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 02:36:43.055474 14209 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 046891eb17b943768c9c410f354b3093 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:36:43.057582 14209 raft_consensus.cc:504] T 00000000000000000000000000000000 P 046891eb17b943768c9c410f354b3093 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "046891eb17b943768c9c410f354b3093" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 42903 } }
I20260812 02:36:43.058037 14209 leader_election.cc:302] T 00000000000000000000000000000000 P 046891eb17b943768c9c410f354b3093 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 046891eb17b943768c9c410f354b3093; no voters:
I20260812 02:36:43.058748 14209 leader_election.cc:288] T 00000000000000000000000000000000 P 046891eb17b943768c9c410f354b3093 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 02:36:43.058877 14215 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 046891eb17b943768c9c410f354b3093 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:36:43.059617 14215 raft_consensus.cc:686] T 00000000000000000000000000000000 P 046891eb17b943768c9c410f354b3093 [term 1 LEADER]: Becoming Leader. State: Replica: 046891eb17b943768c9c410f354b3093, State: Running, Role: LEADER
I20260812 02:36:43.060148 14215 consensus_queue.cc:237] T 00000000000000000000000000000000 P 046891eb17b943768c9c410f354b3093 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "046891eb17b943768c9c410f354b3093" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 42903 } }
I20260812 02:36:43.060877 14209 sys_catalog.cc:564] T 00000000000000000000000000000000 P 046891eb17b943768c9c410f354b3093 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:36:43.063472 14216 sys_catalog.cc:455] T 00000000000000000000000000000000 P 046891eb17b943768c9c410f354b3093 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "046891eb17b943768c9c410f354b3093" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "046891eb17b943768c9c410f354b3093" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 42903 } } }
I20260812 02:36:43.063995 14216 sys_catalog.cc:458] T 00000000000000000000000000000000 P 046891eb17b943768c9c410f354b3093 [sys.catalog]: This master's current role is: LEADER
I20260812 02:36:43.063498 14217 sys_catalog.cc:455] T 00000000000000000000000000000000 P 046891eb17b943768c9c410f354b3093 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 046891eb17b943768c9c410f354b3093. Latest consensus state: current_term: 1 leader_uuid: "046891eb17b943768c9c410f354b3093" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "046891eb17b943768c9c410f354b3093" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 42903 } } }
I20260812 02:36:43.064697 14217 sys_catalog.cc:458] T 00000000000000000000000000000000 P 046891eb17b943768c9c410f354b3093 [sys.catalog]: This master's current role is: LEADER
I20260812 02:36:43.071058 14228 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:36:43.076854 14228 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 02:36:43.086951 14228 catalog_manager.cc:1347] Generated new cluster ID: 3607516bd06f4d37874d104a1851de3e
I20260812 02:36:43.087069 14228 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 02:36:43.104663 14228 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 02:36:43.106602 14228 catalog_manager.cc:1504] Loading token signing keys...
I20260812 02:36:43.115566 14228 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 046891eb17b943768c9c410f354b3093: Generated new TSK 0
I20260812 02:36:43.116644 14228 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260812 02:36:43.151789 14251 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260812 02:36:43.151924 14251 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
W20260812 02:36:43.153309 14250 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:36:43.132391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:42903 (local address 127.0.0.1:46910)
0812 02:36:43.133063 (+ 672us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:43.133084 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:43.133203 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:43.133688 (+ 485us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:43.133692 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:43.133702 (+ 10us) client_negotiation.cc:189] Negotiated authn=JWT
0812 02:36:43.134003 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:43.134013 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:43.135560 (+ 1547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:43.135564 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:43.136657 (+ 1093us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:43.136669 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:43.137708 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:43.137712 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:43.137857 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:43.137889 (+ 32us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 02:36:43.152776 (+ 14887us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35603/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0812 02:36:43.153021 (+ 245us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.7.132.190:42903: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35603/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":523,"thread_start_us":371,"threads_started":1}
I20260812 02:36:43.153298 14212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:43.132405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46910 (local address 127.7.132.190:42903)
0812 02:36:43.132610 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:43.132622 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:43.133173 (+ 551us) server_negotiation.cc:407] Connection header received
0812 02:36:43.133391 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:43.133397 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:43.133489 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:43.133662 (+ 173us) server_negotiation.cc:226] Negotiated authn=JWT
0812 02:36:43.134224 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:43.135397 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:43.136809 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:43.137535 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:43.137674 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:43.138206 (+ 532us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 02:36:43.152536 (+ 14330us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35603/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0812 02:36:43.152972 (+ 436us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:46910: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35603/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":63}
W20260812 02:36:43.153458 14212 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:46910: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35603/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
I20260812 02:36:43.154717 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 14136
I20260812 02:36:43.156002 14204 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:36:43.255359 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 14136
2026-08-12T02:36:43Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (773 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260812 02:36:43.316795 7698 mini_oidc.cc:152] Starting JWKS server
I20260812 02:36:43.318794 7698 webserver.cc:460] Webserver started at https://127.0.0.1:40579/ using document root <none> and password file <none>
I20260812 02:36:43.319022 7698 mini_oidc.cc:189] Starting OIDC Discovery server
I20260812 02:36:43.319612 7698 webserver.cc:460] Webserver started at http://127.0.0.1:34561/ using document root <none> and password file <none>
2026-08-12T02:36:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T02:36:43Z Disabled control of system clock
I20260812 02:36:43.357908 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786502156971023-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786502156971023-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786502156971023-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:39471
--webserver_interface=127.7.132.190
--webserver_port=0
--builtin_ntp_servers=127.7.132.148:41997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.132.190:39471
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40579/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260812 02:36:43.694495 14260 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:36:43.694795 14260 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:36:43.694995 14260 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:36:43.702401 14260 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:36:43.702502 14260 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:36:43.702567 14260 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:36:43.702689 14260 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:36:43.702787 14260 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 02:36:43.702885 14260 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 02:36:43.702953 14260 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:40579/jwks.json
I20260812 02:36:43.712134 14260 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:41997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786502156971023-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786502156971023-7698-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.132.190:39471
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:39471
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40579/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.132.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786502156971023-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:36:43.713477 14260 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:36:43.715382 14260 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:36:43.728166 14266 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:36:43.728120 14267 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:36:43.728451 14269 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:36:43.728940 14260 server_base.cc:1034] running on GCE node
I20260812 02:36:43.729871 14260 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:36:43.731421 14260 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:36:43.732627 14260 hybrid_clock.cc:648] HybridClock initialized: now 1786502203732563 us; error 60 us; skew 500 ppm
I20260812 02:36:43.732995 14260 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 02:36:43.735457 14260 webserver.cc:460] Webserver started at http://127.7.132.190:36937/ using document root <none> and password file <none>
I20260812 02:36:43.736204 14260 fs_manager.cc:359] Metadata directory not provided
I20260812 02:36:43.736295 14260 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:36:43.736670 14260 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:36:43.739745 14260 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786502156971023-7698-0/minicluster-data/master-0/data/instance:
uuid: "a65d604d992645c69a3ce2192cdb4a2e"
format_stamp: "Formatted at 2026-08-12 02:36:43 on dist-test-slave-mn4r"
server_key: "0c33dab68071d984519a08fe031120d0"
server_key_iv: "7bb6c389cb677319fb389f497a233473"
server_key_version: "encryptionkey@0"
I20260812 02:36:43.740569 14260 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786502156971023-7698-0/minicluster-data/master-0/wal/instance:
uuid: "a65d604d992645c69a3ce2192cdb4a2e"
format_stamp: "Formatted at 2026-08-12 02:36:43 on dist-test-slave-mn4r"
server_key: "0c33dab68071d984519a08fe031120d0"
server_key_iv: "7bb6c389cb677319fb389f497a233473"
server_key_version: "encryptionkey@0"
I20260812 02:36:43.745745 14260 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260812 02:36:43.749228 14277 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:36:43.750568 14260 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 02:36:43.750762 14260 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786502156971023-7698-0/minicluster-data/master-0/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786502156971023-7698-0/minicluster-data/master-0/wal
uuid: "a65d604d992645c69a3ce2192cdb4a2e"
format_stamp: "Formatted at 2026-08-12 02:36:43 on dist-test-slave-mn4r"
server_key: "0c33dab68071d984519a08fe031120d0"
server_key_iv: "7bb6c389cb677319fb389f497a233473"
server_key_version: "encryptionkey@0"
I20260812 02:36:43.750995 14260 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786502156971023-7698-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786502156971023-7698-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786502156971023-7698-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:36:43.762254 14260 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:36:43.763047 14260 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:36:43.763363 14260 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:36:43.782464 14260 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.190:39471
I20260812 02:36:43.782483 14335 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.190:39471 every 8 connection(s)
I20260812 02:36:43.783865 14260 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
I20260812 02:36:43.787707 14336 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:36:43.789398 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 14260
I20260812 02:36:43.789636 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786502156971023-7698-0/minicluster-data/master-0/wal/instance
I20260812 02:36:43.790331 7698 external_mini_cluster.cc:1428] Setting key 2619f09caa5bf3ae7bb022d4293b0afa
I20260812 02:36:43.799325 14336 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a65d604d992645c69a3ce2192cdb4a2e: Bootstrap starting.
I20260812 02:36:43.803050 14336 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a65d604d992645c69a3ce2192cdb4a2e: Neither blocks nor log segments found. Creating new log.
I20260812 02:36:43.804363 14336 log.cc:824] T 00000000000000000000000000000000 P a65d604d992645c69a3ce2192cdb4a2e: Log is configured to *not* fsync() on all Append() calls
I20260812 02:36:43.804584 14340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:43.792276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55016 (local address 127.7.132.190:39471)
0812 02:36:43.793295 (+ 1019us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:43.793306 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:43.793349 (+ 43us) server_negotiation.cc:407] Connection header received
0812 02:36:43.795051 (+ 1702us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:43.795076 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:43.795540 (+ 464us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:43.795938 (+ 398us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:43.796551 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:43.797894 (+ 1343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:43.799279 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:43.800028 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:43.800228 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:43.801601 (+ 1373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:43.801613 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:43.801635 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:36:43.802406 (+ 771us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:43.802523 (+ 117us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:36:43.802552 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:43.802685 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:43.802879 (+ 194us) server_negotiation.cc:299] Negotiation successful
0812 02:36:43.803245 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":590,"thread_start_us":360,"threads_started":1}
I20260812 02:36:43.808039 14336 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a65d604d992645c69a3ce2192cdb4a2e: No bootstrap required, opened a new log
I20260812 02:36:43.809221 7698 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 02:36:43.817092 14336 raft_consensus.cc:348] T 00000000000000000000000000000000 P a65d604d992645c69a3ce2192cdb4a2e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a65d604d992645c69a3ce2192cdb4a2e" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 39471 } }
I20260812 02:36:43.817435 14336 raft_consensus.cc:374] T 00000000000000000000000000000000 P a65d604d992645c69a3ce2192cdb4a2e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:36:43.817508 14336 raft_consensus.cc:729] T 00000000000000000000000000000000 P a65d604d992645c69a3ce2192cdb4a2e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a65d604d992645c69a3ce2192cdb4a2e, State: Initialized, Role: FOLLOWER
I20260812 02:36:43.818147 14336 consensus_queue.cc:260] T 00000000000000000000000000000000 P a65d604d992645c69a3ce2192cdb4a2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a65d604d992645c69a3ce2192cdb4a2e" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 39471 } }
I20260812 02:36:43.818481 14336 raft_consensus.cc:388] T 00000000000000000000000000000000 P a65d604d992645c69a3ce2192cdb4a2e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 02:36:43.818651 14336 raft_consensus.cc:482] T 00000000000000000000000000000000 P a65d604d992645c69a3ce2192cdb4a2e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 02:36:43.818830 14336 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a65d604d992645c69a3ce2192cdb4a2e [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:36:43.820966 14336 raft_consensus.cc:504] T 00000000000000000000000000000000 P a65d604d992645c69a3ce2192cdb4a2e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a65d604d992645c69a3ce2192cdb4a2e" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 39471 } }
I20260812 02:36:43.821460 14336 leader_election.cc:302] T 00000000000000000000000000000000 P a65d604d992645c69a3ce2192cdb4a2e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a65d604d992645c69a3ce2192cdb4a2e; no voters:
I20260812 02:36:43.822137 14336 leader_election.cc:288] T 00000000000000000000000000000000 P a65d604d992645c69a3ce2192cdb4a2e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 02:36:43.822288 14342 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a65d604d992645c69a3ce2192cdb4a2e [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:36:43.823011 14342 raft_consensus.cc:686] T 00000000000000000000000000000000 P a65d604d992645c69a3ce2192cdb4a2e [term 1 LEADER]: Becoming Leader. State: Replica: a65d604d992645c69a3ce2192cdb4a2e, State: Running, Role: LEADER
I20260812 02:36:43.823535 14342 consensus_queue.cc:237] T 00000000000000000000000000000000 P a65d604d992645c69a3ce2192cdb4a2e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a65d604d992645c69a3ce2192cdb4a2e" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 39471 } }
I20260812 02:36:43.824292 14336 sys_catalog.cc:564] T 00000000000000000000000000000000 P a65d604d992645c69a3ce2192cdb4a2e [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:36:43.826455 14345 sys_catalog.cc:455] T 00000000000000000000000000000000 P a65d604d992645c69a3ce2192cdb4a2e [sys.catalog]: SysCatalogTable state changed. Reason: New leader a65d604d992645c69a3ce2192cdb4a2e. Latest consensus state: current_term: 1 leader_uuid: "a65d604d992645c69a3ce2192cdb4a2e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a65d604d992645c69a3ce2192cdb4a2e" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 39471 } } }
I20260812 02:36:43.826833 14345 sys_catalog.cc:458] T 00000000000000000000000000000000 P a65d604d992645c69a3ce2192cdb4a2e [sys.catalog]: This master's current role is: LEADER
I20260812 02:36:43.827293 14343 sys_catalog.cc:455] T 00000000000000000000000000000000 P a65d604d992645c69a3ce2192cdb4a2e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a65d604d992645c69a3ce2192cdb4a2e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a65d604d992645c69a3ce2192cdb4a2e" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 39471 } } }
I20260812 02:36:43.827651 14343 sys_catalog.cc:458] T 00000000000000000000000000000000 P a65d604d992645c69a3ce2192cdb4a2e [sys.catalog]: This master's current role is: LEADER
I20260812 02:36:43.830412 14349 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:36:43.835840 14349 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 02:36:43.844957 14349 catalog_manager.cc:1347] Generated new cluster ID: 6322e5b97dcd431bac8fefed73634d58
I20260812 02:36:43.845079 14349 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 02:36:43.859113 14349 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 02:36:43.860984 14349 catalog_manager.cc:1504] Loading token signing keys...
I20260812 02:36:43.876385 14349 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a65d604d992645c69a3ce2192cdb4a2e: Generated new TSK 0
I20260812 02:36:43.877544 14349 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 02:36:43.906878 14340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:43.898041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55030 (local address 127.7.132.190:39471)
0812 02:36:43.898266 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:43.898278 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:43.898743 (+ 465us) server_negotiation.cc:407] Connection header received
0812 02:36:43.898972 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:43.898978 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:43.899080 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:43.899266 (+ 186us) server_negotiation.cc:226] Negotiated authn=JWT
0812 02:36:43.899700 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:43.900858 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:43.902340 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:43.903075 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:43.903216 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:43.903688 (+ 472us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 02:36:43.906175 (+ 2487us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40579/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0812 02:36:43.906537 (+ 362us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:55030: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40579/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":84}
W20260812 02:36:43.907032 14340 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:55030: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40579/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260812 02:36:43.908571 14371 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:36:43.897895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:39471 (local address 127.0.0.1:55030)
0812 02:36:43.898636 (+ 741us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:43.898654 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:43.898780 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:43.899268 (+ 488us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:43.899272 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:43.899281 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0812 02:36:43.899553 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:43.899561 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:43.900997 (+ 1436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:43.901001 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:43.902193 (+ 1192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:43.902206 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:43.903274 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:43.903279 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:43.903395 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:43.903427 (+ 32us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 02:36:43.906420 (+ 2993us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40579/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0812 02:36:43.908145 (+ 1725us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.7.132.190:39471: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40579/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":577,"mutex_wait_us":1413,"thread_start_us":430,"threads_started":1}
I20260812 02:36:43.909716 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 14260
I20260812 02:36:43.911103 14331 generic_service.cc:196] Checking for leaks (request via RPC)
E20260812 02:36:43.922649 14372 webserver.cc:533] Webserver: sslize failed:
E20260812 02:36:43.922812 14372 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260812 02:36:44.005764 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 14260
2026-08-12T02:36:44Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (752 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-12T02:36:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T02:36:44Z Disabled control of system clock
I20260812 02:36:44.104539 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786502156971023-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786502156971023-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786502156971023-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:38765
--webserver_interface=127.7.132.190
--webserver_port=0
--builtin_ntp_servers=127.7.132.148:37101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.132.190:38765
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786502156971023-7698-0/testchainca.cert with env {}
W20260812 02:36:44.448504 14379 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:36:44.448820 14379 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:36:44.448915 14379 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:36:44.456185 14379 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:36:44.456296 14379 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:36:44.456337 14379 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:36:44.456387 14379 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:36:44.456424 14379 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 02:36:44.456489 14379 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 02:36:44.456537 14379 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260812 02:36:44.467124 14379 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:37101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786502156971023-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786502156971023-7698-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.132.190:38765
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:38765
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.132.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786502156971023-7698-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786502156971023-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:36:44.468402 14379 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:36:44.469718 14379 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:36:44.482178 14386 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:36:44.482167 14389 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:36:44.482203 14387 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:36:44.483064 14379 server_base.cc:1034] running on GCE node
I20260812 02:36:44.483924 14379 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:36:44.485361 14379 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:36:44.486747 14379 hybrid_clock.cc:648] HybridClock initialized: now 1786502204486665 us; error 69 us; skew 500 ppm
I20260812 02:36:44.487167 14379 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 02:36:44.489562 14379 webserver.cc:460] Webserver started at http://127.7.132.190:41205/ using document root <none> and password file <none>
I20260812 02:36:44.490365 14379 fs_manager.cc:359] Metadata directory not provided
I20260812 02:36:44.490512 14379 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:36:44.490847 14379 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:36:44.493831 14379 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786502156971023-7698-0/minicluster-data/master-0/data/instance:
uuid: "e50e4ffe849e4f7a9e5ab2e48b107826"
format_stamp: "Formatted at 2026-08-12 02:36:44 on dist-test-slave-mn4r"
server_key: "3913a54b2a30afe64af6a5fc36680ff8"
server_key_iv: "d508a939eb202dc872be9f2867a58831"
server_key_version: "encryptionkey@0"
I20260812 02:36:44.494768 14379 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786502156971023-7698-0/minicluster-data/master-0/wal/instance:
uuid: "e50e4ffe849e4f7a9e5ab2e48b107826"
format_stamp: "Formatted at 2026-08-12 02:36:44 on dist-test-slave-mn4r"
server_key: "3913a54b2a30afe64af6a5fc36680ff8"
server_key_iv: "d508a939eb202dc872be9f2867a58831"
server_key_version: "encryptionkey@0"
I20260812 02:36:44.500242 14379 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260812 02:36:44.503873 14396 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:36:44.505276 14379 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 02:36:44.505475 14379 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786502156971023-7698-0/minicluster-data/master-0/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786502156971023-7698-0/minicluster-data/master-0/wal
uuid: "e50e4ffe849e4f7a9e5ab2e48b107826"
format_stamp: "Formatted at 2026-08-12 02:36:44 on dist-test-slave-mn4r"
server_key: "3913a54b2a30afe64af6a5fc36680ff8"
server_key_iv: "d508a939eb202dc872be9f2867a58831"
server_key_version: "encryptionkey@0"
I20260812 02:36:44.505722 14379 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786502156971023-7698-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786502156971023-7698-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786502156971023-7698-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:36:44.522845 14379 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:36:44.523691 14379 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:36:44.523988 14379 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:36:44.542734 14379 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.190:38765
I20260812 02:36:44.542771 14458 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.190:38765 every 8 connection(s)
I20260812 02:36:44.544131 14379 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
I20260812 02:36:44.547330 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 14379
I20260812 02:36:44.547533 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786502156971023-7698-0/minicluster-data/master-0/wal/instance
I20260812 02:36:44.548415 7698 external_mini_cluster.cc:1428] Setting key 13398f61001a85cc60dc8fd61c4225d2
I20260812 02:36:44.548480 14459 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:36:44.562350 14459 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e50e4ffe849e4f7a9e5ab2e48b107826: Bootstrap starting.
I20260812 02:36:44.562661 14463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:44.550465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36458 (local address 127.7.132.190:38765)
0812 02:36:44.551497 (+ 1032us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:44.551515 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:44.551558 (+ 43us) server_negotiation.cc:407] Connection header received
0812 02:36:44.553224 (+ 1666us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:44.553247 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:44.553717 (+ 470us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:44.554051 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:44.554737 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:44.556158 (+ 1421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:44.557990 (+ 1832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:44.558756 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:44.558893 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:44.560181 (+ 1288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:44.560188 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:44.560204 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:36:44.560761 (+ 557us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:44.560836 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:36:44.560857 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:44.560951 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:44.561201 (+ 250us) server_negotiation.cc:299] Negotiation successful
0812 02:36:44.561555 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":613,"thread_start_us":352,"threads_started":1}
I20260812 02:36:44.565668 14459 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e50e4ffe849e4f7a9e5ab2e48b107826: Neither blocks nor log segments found. Creating new log.
I20260812 02:36:44.566498 7698 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 02:36:44.566921 14459 log.cc:824] T 00000000000000000000000000000000 P e50e4ffe849e4f7a9e5ab2e48b107826: Log is configured to *not* fsync() on all Append() calls
I20260812 02:36:44.569909 14459 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e50e4ffe849e4f7a9e5ab2e48b107826: No bootstrap required, opened a new log
I20260812 02:36:44.577217 14459 raft_consensus.cc:348] T 00000000000000000000000000000000 P e50e4ffe849e4f7a9e5ab2e48b107826 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e50e4ffe849e4f7a9e5ab2e48b107826" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 38765 } }
I20260812 02:36:44.577580 14459 raft_consensus.cc:374] T 00000000000000000000000000000000 P e50e4ffe849e4f7a9e5ab2e48b107826 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:36:44.577710 14459 raft_consensus.cc:729] T 00000000000000000000000000000000 P e50e4ffe849e4f7a9e5ab2e48b107826 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e50e4ffe849e4f7a9e5ab2e48b107826, State: Initialized, Role: FOLLOWER
I20260812 02:36:44.578364 14459 consensus_queue.cc:260] T 00000000000000000000000000000000 P e50e4ffe849e4f7a9e5ab2e48b107826 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e50e4ffe849e4f7a9e5ab2e48b107826" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 38765 } }
I20260812 02:36:44.578606 14459 raft_consensus.cc:388] T 00000000000000000000000000000000 P e50e4ffe849e4f7a9e5ab2e48b107826 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 02:36:44.578761 14459 raft_consensus.cc:482] T 00000000000000000000000000000000 P e50e4ffe849e4f7a9e5ab2e48b107826 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 02:36:44.578929 14459 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e50e4ffe849e4f7a9e5ab2e48b107826 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:36:44.581302 14459 raft_consensus.cc:504] T 00000000000000000000000000000000 P e50e4ffe849e4f7a9e5ab2e48b107826 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e50e4ffe849e4f7a9e5ab2e48b107826" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 38765 } }
I20260812 02:36:44.581820 14459 leader_election.cc:302] T 00000000000000000000000000000000 P e50e4ffe849e4f7a9e5ab2e48b107826 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e50e4ffe849e4f7a9e5ab2e48b107826; no voters:
I20260812 02:36:44.582535 14459 leader_election.cc:288] T 00000000000000000000000000000000 P e50e4ffe849e4f7a9e5ab2e48b107826 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 02:36:44.582705 14466 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e50e4ffe849e4f7a9e5ab2e48b107826 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:36:44.583480 14466 raft_consensus.cc:686] T 00000000000000000000000000000000 P e50e4ffe849e4f7a9e5ab2e48b107826 [term 1 LEADER]: Becoming Leader. State: Replica: e50e4ffe849e4f7a9e5ab2e48b107826, State: Running, Role: LEADER
I20260812 02:36:44.584051 14466 consensus_queue.cc:237] T 00000000000000000000000000000000 P e50e4ffe849e4f7a9e5ab2e48b107826 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e50e4ffe849e4f7a9e5ab2e48b107826" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 38765 } }
I20260812 02:36:44.585001 14459 sys_catalog.cc:564] T 00000000000000000000000000000000 P e50e4ffe849e4f7a9e5ab2e48b107826 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:36:44.587379 14467 sys_catalog.cc:455] T 00000000000000000000000000000000 P e50e4ffe849e4f7a9e5ab2e48b107826 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e50e4ffe849e4f7a9e5ab2e48b107826" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e50e4ffe849e4f7a9e5ab2e48b107826" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 38765 } } }
I20260812 02:36:44.587849 14467 sys_catalog.cc:458] T 00000000000000000000000000000000 P e50e4ffe849e4f7a9e5ab2e48b107826 [sys.catalog]: This master's current role is: LEADER
I20260812 02:36:44.589263 14468 sys_catalog.cc:455] T 00000000000000000000000000000000 P e50e4ffe849e4f7a9e5ab2e48b107826 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e50e4ffe849e4f7a9e5ab2e48b107826. Latest consensus state: current_term: 1 leader_uuid: "e50e4ffe849e4f7a9e5ab2e48b107826" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e50e4ffe849e4f7a9e5ab2e48b107826" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 38765 } } }
I20260812 02:36:44.589547 14468 sys_catalog.cc:458] T 00000000000000000000000000000000 P e50e4ffe849e4f7a9e5ab2e48b107826 [sys.catalog]: This master's current role is: LEADER
I20260812 02:36:44.590070 14478 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:36:44.595022 14478 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 02:36:44.603567 14478 catalog_manager.cc:1347] Generated new cluster ID: 78bc4739619a4100b4171f804ab5c4aa
I20260812 02:36:44.603684 14478 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 02:36:44.616521 14478 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 02:36:44.618429 14478 catalog_manager.cc:1504] Loading token signing keys...
I20260812 02:36:44.631774 14478 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e50e4ffe849e4f7a9e5ab2e48b107826: Generated new TSK 0
I20260812 02:36:44.632865 14478 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 02:36:45.083813 14463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:44.650714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36460 (local address 127.7.132.190:38765)
0812 02:36:44.650928 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:44.650935 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:44.651525 (+ 590us) server_negotiation.cc:407] Connection header received
0812 02:36:44.651746 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:44.651752 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:44.651854 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:44.652034 (+ 180us) server_negotiation.cc:226] Negotiated authn=JWT
0812 02:36:44.652541 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:44.653680 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:44.655338 (+ 1658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:44.656059 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:44.656213 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:44.656707 (+ 494us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 02:36:45.083241 (+426534us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0812 02:36:45.083587 (+ 346us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:36460: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":68}
W20260812 02:36:45.083994 14463 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:36460: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260812 02:36:45.084173 14494 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:36:44.650711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:38765 (local address 127.0.0.1:36460)
0812 02:36:44.651409 (+ 698us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:44.651426 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:44.651560 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:44.652056 (+ 496us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:44.652060 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:44.652071 (+ 11us) client_negotiation.cc:189] Negotiated authn=JWT
0812 02:36:44.652366 (+ 295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:44.652374 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:44.653862 (+ 1488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:44.653866 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:44.655179 (+ 1313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:44.655201 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:44.656240 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:44.656244 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:44.656387 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:44.656430 (+ 43us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 02:36:45.083536 (+427106us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0812 02:36:45.083793 (+ 257us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.7.132.190:38765: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":541,"thread_start_us":383,"threads_started":1}
I20260812 02:36:45.085747 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 14379
I20260812 02:36:45.086879 14454 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:36:45.190408 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 14379
2026-08-12T02:36:45Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (1163 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (432 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (422 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502156971023-7698-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 02:36:46 dist-test-slave-mn4r krb5kdc[14512](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 02:36:46 dist-test-slave-mn4r krb5kdc[14512](info): set up 2 sockets
Aug 12 02:36:46 dist-test-slave-mn4r krb5kdc[14512](info): commencing operation
krb5kdc: starting...
W20260812 02:36:47.249418 7698 mini_kdc.cc:121] Time spent starting KDC: real 1.159s user 0.000s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502156971023-7698-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502156971023-7698-0/krb5kdc/test-user.keytab.
Aug 12 02:36:47 dist-test-slave-mn4r krb5kdc[14512](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502207, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T02:36:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T02:36:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.132.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
I20260812 02:36:47.501443 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502156971023-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502156971023-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502156971023-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:34501
--webserver_interface=127.7.132.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:39719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.132.190:34501
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:36:47.858686 14529 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:36:47.858994 14529 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:36:47.859082 14529 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:36:47.866475 14529 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:36:47.866592 14529 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:36:47.866647 14529 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:36:47.866704 14529 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:36:47.866742 14529 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:36:47.876771 14529 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:39719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502156971023-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502156971023-7698-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.132.190:34501
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:34501
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.132.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502156971023-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:36:47.878116 14529 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:36:47.879442 14529 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:36:47.891703 14535 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:36:47.891738 14538 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:36:47.892210 14536 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:36:47.892855 14529 server_base.cc:1034] running on GCE node
I20260812 02:36:47.893689 14529 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:36:47.895237 14529 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:36:47.896495 14529 hybrid_clock.cc:648] HybridClock initialized: now 1786502207896444 us; error 36 us; skew 500 ppm
I20260812 02:36:47.896876 14529 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:36:47 dist-test-slave-mn4r krb5kdc[14512](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502207, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:36:47.900071 14529 init.cc:377] Logged in from keytab as kudu/127.7.132.190@KRBTEST.COM (short username kudu)
I20260812 02:36:47.901744 14529 webserver.cc:460] Webserver started at http://127.7.132.190:38033/ using document root <none> and password file <none>
I20260812 02:36:47.902554 14529 fs_manager.cc:359] Metadata directory not provided
I20260812 02:36:47.902714 14529 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:36:47.903047 14529 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:36:47.906204 14529 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502156971023-7698-0/minicluster-data/master-0/data/instance:
uuid: "63af8c45f60b4261936202f355fff0cb"
format_stamp: "Formatted at 2026-08-12 02:36:47 on dist-test-slave-mn4r"
server_key: "548a016e543abca08f8d3d460bb71d58"
server_key_iv: "a475e81279fa2e69ff08a4ba4c0c258b"
server_key_version: "encryptionkey@0"
I20260812 02:36:47.907091 14529 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502156971023-7698-0/minicluster-data/master-0/wal/instance:
uuid: "63af8c45f60b4261936202f355fff0cb"
format_stamp: "Formatted at 2026-08-12 02:36:47 on dist-test-slave-mn4r"
server_key: "548a016e543abca08f8d3d460bb71d58"
server_key_iv: "a475e81279fa2e69ff08a4ba4c0c258b"
server_key_version: "encryptionkey@0"
I20260812 02:36:47.913159 14529 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260812 02:36:47.917263 14545 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:36:47.918845 14529 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 02:36:47.919059 14529 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502156971023-7698-0/minicluster-data/master-0/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502156971023-7698-0/minicluster-data/master-0/wal
uuid: "63af8c45f60b4261936202f355fff0cb"
format_stamp: "Formatted at 2026-08-12 02:36:47 on dist-test-slave-mn4r"
server_key: "548a016e543abca08f8d3d460bb71d58"
server_key_iv: "a475e81279fa2e69ff08a4ba4c0c258b"
server_key_version: "encryptionkey@0"
I20260812 02:36:47.919291 14529 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502156971023-7698-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502156971023-7698-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502156971023-7698-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:36:47.933416 14529 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:36:47.938474 14529 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:36:47.938815 14529 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:36:47.958189 14529 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.190:34501
I20260812 02:36:47.958278 14608 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.190:34501 every 8 connection(s)
I20260812 02:36:47.959652 14529 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
I20260812 02:36:47.963469 14609 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:36:47.967007 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 14529
I20260812 02:36:47.967242 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502156971023-7698-0/minicluster-data/master-0/wal/instance
I20260812 02:36:47.968151 7698 external_mini_cluster.cc:1428] Setting key 7ea02b447e10968aa5a7176c219d3772
I20260812 02:36:47.977118 14609 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63af8c45f60b4261936202f355fff0cb: Bootstrap starting.
I20260812 02:36:47.981087 14609 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 63af8c45f60b4261936202f355fff0cb: Neither blocks nor log segments found. Creating new log.
I20260812 02:36:47.982522 14609 log.cc:824] T 00000000000000000000000000000000 P 63af8c45f60b4261936202f355fff0cb: Log is configured to *not* fsync() on all Append() calls
Aug 12 02:36:47 dist-test-slave-mn4r krb5kdc[14512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502207, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 02:36:47.985737 14609 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63af8c45f60b4261936202f355fff0cb: No bootstrap required, opened a new log
I20260812 02:36:47.992318 14612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:47.970382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44840 (local address 127.7.132.190:34501)
0812 02:36:47.971193 (+ 811us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:47.971205 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:47.971248 (+ 43us) server_negotiation.cc:407] Connection header received
0812 02:36:47.972886 (+ 1638us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:47.972909 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:47.973333 (+ 424us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:47.973665 (+ 332us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:47.975490 (+ 1825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:47.976801 (+ 1311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:47.978551 (+ 1750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:47.979309 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:47.979446 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:47.984649 (+ 5203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:47.984658 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:47.984672 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:47.984711 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:47.987967 (+ 3256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:47.988691 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:47.988698 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:47.988705 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:47.988884 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:47.989379 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:47.989392 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:47.989394 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:47.989942 (+ 548us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:47.990329 (+ 387us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:47.990794 (+ 465us) server_negotiation.cc:299] Negotiation successful
0812 02:36:47.991234 (+ 440us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":597,"thread_start_us":393,"threads_started":1}
I20260812 02:36:47.994177 14609 raft_consensus.cc:348] T 00000000000000000000000000000000 P 63af8c45f60b4261936202f355fff0cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63af8c45f60b4261936202f355fff0cb" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 34501 } }
I20260812 02:36:47.994624 14609 raft_consensus.cc:374] T 00000000000000000000000000000000 P 63af8c45f60b4261936202f355fff0cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:36:47.994747 14609 raft_consensus.cc:729] T 00000000000000000000000000000000 P 63af8c45f60b4261936202f355fff0cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63af8c45f60b4261936202f355fff0cb, State: Initialized, Role: FOLLOWER
I20260812 02:36:47.995451 14609 consensus_queue.cc:260] T 00000000000000000000000000000000 P 63af8c45f60b4261936202f355fff0cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63af8c45f60b4261936202f355fff0cb" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 34501 } }
I20260812 02:36:47.995725 14609 raft_consensus.cc:388] T 00000000000000000000000000000000 P 63af8c45f60b4261936202f355fff0cb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 02:36:47.995832 14609 raft_consensus.cc:482] T 00000000000000000000000000000000 P 63af8c45f60b4261936202f355fff0cb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 02:36:47.995955 14609 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 63af8c45f60b4261936202f355fff0cb [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:36:47.998036 14609 raft_consensus.cc:504] T 00000000000000000000000000000000 P 63af8c45f60b4261936202f355fff0cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63af8c45f60b4261936202f355fff0cb" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 34501 } }
I20260812 02:36:47.998559 14609 leader_election.cc:302] T 00000000000000000000000000000000 P 63af8c45f60b4261936202f355fff0cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 63af8c45f60b4261936202f355fff0cb; no voters:
I20260812 02:36:47.999079 14609 leader_election.cc:288] T 00000000000000000000000000000000 P 63af8c45f60b4261936202f355fff0cb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 02:36:47.999221 14614 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 63af8c45f60b4261936202f355fff0cb [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:36:47.999930 14614 raft_consensus.cc:686] T 00000000000000000000000000000000 P 63af8c45f60b4261936202f355fff0cb [term 1 LEADER]: Becoming Leader. State: Replica: 63af8c45f60b4261936202f355fff0cb, State: Running, Role: LEADER
I20260812 02:36:48.000473 14614 consensus_queue.cc:237] T 00000000000000000000000000000000 P 63af8c45f60b4261936202f355fff0cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63af8c45f60b4261936202f355fff0cb" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 34501 } }
I20260812 02:36:48.001343 14609 sys_catalog.cc:564] T 00000000000000000000000000000000 P 63af8c45f60b4261936202f355fff0cb [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:36:48.003728 14616 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63af8c45f60b4261936202f355fff0cb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 63af8c45f60b4261936202f355fff0cb. Latest consensus state: current_term: 1 leader_uuid: "63af8c45f60b4261936202f355fff0cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63af8c45f60b4261936202f355fff0cb" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 34501 } } }
I20260812 02:36:48.004196 14616 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63af8c45f60b4261936202f355fff0cb [sys.catalog]: This master's current role is: LEADER
I20260812 02:36:48.006273 14615 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63af8c45f60b4261936202f355fff0cb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "63af8c45f60b4261936202f355fff0cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63af8c45f60b4261936202f355fff0cb" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 34501 } } }
I20260812 02:36:48.006543 14615 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63af8c45f60b4261936202f355fff0cb [sys.catalog]: This master's current role is: LEADER
I20260812 02:36:48.007054 14622 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:36:48.012126 14622 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 02:36:48.021258 14622 catalog_manager.cc:1347] Generated new cluster ID: c57c37ca041a4379a491079b5e384691
I20260812 02:36:48.021371 14622 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 02:36:48.031363 14622 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 02:36:48.032738 14622 catalog_manager.cc:1504] Loading token signing keys...
I20260812 02:36:48.040565 14622 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 63af8c45f60b4261936202f355fff0cb: Generated new TSK 0
I20260812 02:36:48.041479 14622 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.132.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.129.keytab.
Entry for principal kudu/127.7.132.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.129.keytab.
I20260812 02:36:48.105436 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502156971023-7698-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502156971023-7698-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502156971023-7698-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502156971023-7698-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.129:0
--local_ip_for_outbound_sockets=127.7.132.129
--webserver_interface=127.7.132.129
--webserver_port=0
--tserver_master_addrs=127.7.132.190:34501
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.129.keytab
--principal=kudu/127.7.132.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:39719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:36:48.452231 14639 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:36:48.452518 14639 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:36:48.452680 14639 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:36:48.460083 14639 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:36:48.460191 14639 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:36:48.460358 14639 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.129
I20260812 02:36:48.470352 14639 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:39719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502156971023-7698-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502156971023-7698-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.129.keytab
--principal=kudu/127.7.132.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502156971023-7698-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.132.129
--webserver_port=0
--tserver_master_addrs=127.7.132.190:34501
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.129
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502156971023-7698-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:36:48.471736 14639 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:36:48.473148 14639 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:36:48.486931 14644 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:36:48.486902 14647 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:36:48.487272 14645 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:36:48.487890 14639 server_base.cc:1034] running on GCE node
I20260812 02:36:48.488651 14639 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:36:48.489624 14639 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:36:48.490883 14639 hybrid_clock.cc:648] HybridClock initialized: now 1786502208490832 us; error 42 us; skew 500 ppm
I20260812 02:36:48.491259 14639 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:36:48 dist-test-slave-mn4r krb5kdc[14512](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502208, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:36:48.494745 14639 init.cc:377] Logged in from keytab as kudu/127.7.132.129@KRBTEST.COM (short username kudu)
I20260812 02:36:48.496284 14639 webserver.cc:460] Webserver started at http://127.7.132.129:45531/ using document root <none> and password file <none>
I20260812 02:36:48.497015 14639 fs_manager.cc:359] Metadata directory not provided
I20260812 02:36:48.497169 14639 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:36:48.497500 14639 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:36:48.500630 14639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502156971023-7698-0/minicluster-data/ts-0/data/instance:
uuid: "e36a7ee108b64961ac9de10723b84a88"
format_stamp: "Formatted at 2026-08-12 02:36:48 on dist-test-slave-mn4r"
server_key: "365f83cdae7358533425bf46d222c8f3"
server_key_iv: "b9f2e37b44baf4d494741d63aab1842c"
server_key_version: "encryptionkey@0"
I20260812 02:36:48.501538 14639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502156971023-7698-0/minicluster-data/ts-0/wal/instance:
uuid: "e36a7ee108b64961ac9de10723b84a88"
format_stamp: "Formatted at 2026-08-12 02:36:48 on dist-test-slave-mn4r"
server_key: "365f83cdae7358533425bf46d222c8f3"
server_key_iv: "b9f2e37b44baf4d494741d63aab1842c"
server_key_version: "encryptionkey@0"
I20260812 02:36:48.507037 14639 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260812 02:36:48.510772 14657 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:36:48.512205 14639 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 02:36:48.512379 14639 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502156971023-7698-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502156971023-7698-0/minicluster-data/ts-0/wal
uuid: "e36a7ee108b64961ac9de10723b84a88"
format_stamp: "Formatted at 2026-08-12 02:36:48 on dist-test-slave-mn4r"
server_key: "365f83cdae7358533425bf46d222c8f3"
server_key_iv: "b9f2e37b44baf4d494741d63aab1842c"
server_key_version: "encryptionkey@0"
I20260812 02:36:48.512590 14639 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502156971023-7698-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502156971023-7698-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502156971023-7698-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:36:48.544021 14639 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:36:48.548489 14639 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:36:48.548799 14639 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:36:48.549816 14639 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:36:48.551189 14639 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:36:48.551291 14639 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:36:48.551440 14639 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:36:48.551510 14639 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:36:48.584920 14639 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.129:42989
I20260812 02:36:48.584971 14800 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.129:42989 every 8 connection(s)
I20260812 02:36:48.586328 14639 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502156971023-7698-0/minicluster-data/ts-0/data/info.pb
I20260812 02:36:48.591425 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 14639
I20260812 02:36:48.591655 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502156971023-7698-0/minicluster-data/ts-0/wal/instance
I20260812 02:36:48.592324 7698 external_mini_cluster.cc:1428] Setting key 1c75a9e7845972791e0f956cf808e2d9
Aug 12 02:36:48 dist-test-slave-mn4r krb5kdc[14512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502208, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.129@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 02:36:48.608564 14805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:48.588911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:35537 (local address 127.7.132.190:34501)
0812 02:36:48.589352 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:48.589356 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:48.590446 (+ 1090us) server_negotiation.cc:407] Connection header received
0812 02:36:48.592360 (+ 1914us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:48.592365 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:48.592459 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:48.592581 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:48.594295 (+ 1714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:48.595378 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:48.596952 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:48.597678 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:48.597832 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:48.601799 (+ 3967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:48.601808 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:48.601815 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:48.601864 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:48.605025 (+ 3161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:48.606035 (+ 1010us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:48.606039 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:48.606041 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:48.606238 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:48.606741 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:48.606745 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:48.606746 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:48.607006 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:48.607119 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:48.608112 (+ 993us) server_negotiation.cc:299] Negotiation successful
0812 02:36:48.608293 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":229,"threads_started":1}
I20260812 02:36:48.610342 14806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:48.589522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:34501 (local address 127.7.132.129:35537)
0812 02:36:48.590289 (+ 767us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:48.590336 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:48.592088 (+ 1752us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:48.592826 (+ 738us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:48.592833 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:48.593474 (+ 641us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:36:48.594073 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:48.594124 (+ 51us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:48.595546 (+ 1422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:48.595550 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:48.596799 (+ 1249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:48.596811 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:48.597944 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:48.597950 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:48.598241 (+ 291us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:48.598987 (+ 746us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:36:48.599019 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:36:48.601509 (+ 2490us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:36:48.605185 (+ 3676us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:48.605197 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:48.605218 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:48.605903 (+ 685us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:48.606350 (+ 447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:48.606358 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:48.606364 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:48.606622 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:48.607142 (+ 520us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:36:48.607153 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:36:48.607695 (+ 542us) client_negotiation.cc:769] Sending connection context
0812 02:36:48.608017 (+ 322us) client_negotiation.cc:240] Negotiation successful
0812 02:36:48.608643 (+ 626us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":289,"threads_started":1}
I20260812 02:36:48.612139 14801 heartbeater.cc:347] Connected to a master server at 127.7.132.190:34501
I20260812 02:36:48.612512 14801 heartbeater.cc:464] Registering TS with master...
I20260812 02:36:48.613314 14801 heartbeater.cc:511] Master 127.7.132.190:34501 requested a full tablet report, sending...
I20260812 02:36:48.615921 14566 ts_manager.cc:194] Registered new tserver with Master: e36a7ee108b64961ac9de10723b84a88 (127.7.132.129:42989)
I20260812 02:36:48.617604 14566 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.129@KRBTEST.COM'} at 127.7.132.129:35537
WARNING: no policy specified for kudu/127.7.132.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.130.keytab.
Entry for principal kudu/127.7.132.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.130.keytab.
I20260812 02:36:48.654511 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502156971023-7698-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502156971023-7698-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502156971023-7698-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502156971023-7698-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.130:0
--local_ip_for_outbound_sockets=127.7.132.130
--webserver_interface=127.7.132.130
--webserver_port=0
--tserver_master_addrs=127.7.132.190:34501
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.130.keytab
--principal=kudu/127.7.132.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:39719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:36:49.006141 14809 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:36:49.006479 14809 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:36:49.006587 14809 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:36:49.014061 14809 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:36:49.014261 14809 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:36:49.014386 14809 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.130
I20260812 02:36:49.024905 14809 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:39719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502156971023-7698-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502156971023-7698-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.130.keytab
--principal=kudu/127.7.132.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502156971023-7698-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.132.130
--webserver_port=0
--tserver_master_addrs=127.7.132.190:34501
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.130
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502156971023-7698-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:36:49.026330 14809 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:36:49.027784 14809 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:36:49.041558 14817 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:36:49.041600 14815 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:36:49.041523 14814 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:36:49.042280 14809 server_base.cc:1034] running on GCE node
I20260812 02:36:49.042891 14809 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:36:49.043846 14809 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:36:49.045058 14809 hybrid_clock.cc:648] HybridClock initialized: now 1786502209044988 us; error 63 us; skew 500 ppm
I20260812 02:36:49.045421 14809 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:36:49 dist-test-slave-mn4r krb5kdc[14512](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502209, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:36:49.048832 14809 init.cc:377] Logged in from keytab as kudu/127.7.132.130@KRBTEST.COM (short username kudu)
I20260812 02:36:49.050320 14809 webserver.cc:460] Webserver started at http://127.7.132.130:38025/ using document root <none> and password file <none>
I20260812 02:36:49.051020 14809 fs_manager.cc:359] Metadata directory not provided
I20260812 02:36:49.051128 14809 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:36:49.051393 14809 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:36:49.054448 14809 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502156971023-7698-0/minicluster-data/ts-1/data/instance:
uuid: "a28fd210b5574adc98317f2ed99fa761"
format_stamp: "Formatted at 2026-08-12 02:36:49 on dist-test-slave-mn4r"
server_key: "599be05dba803c605744ee6daf66d062"
server_key_iv: "e7c067da93624478e2dfc0b8cf2afa87"
server_key_version: "encryptionkey@0"
I20260812 02:36:49.055217 14809 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502156971023-7698-0/minicluster-data/ts-1/wal/instance:
uuid: "a28fd210b5574adc98317f2ed99fa761"
format_stamp: "Formatted at 2026-08-12 02:36:49 on dist-test-slave-mn4r"
server_key: "599be05dba803c605744ee6daf66d062"
server_key_iv: "e7c067da93624478e2dfc0b8cf2afa87"
server_key_version: "encryptionkey@0"
I20260812 02:36:49.060539 14809 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 02:36:49.064167 14825 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:36:49.065589 14809 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 02:36:49.065789 14809 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502156971023-7698-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502156971023-7698-0/minicluster-data/ts-1/wal
uuid: "a28fd210b5574adc98317f2ed99fa761"
format_stamp: "Formatted at 2026-08-12 02:36:49 on dist-test-slave-mn4r"
server_key: "599be05dba803c605744ee6daf66d062"
server_key_iv: "e7c067da93624478e2dfc0b8cf2afa87"
server_key_version: "encryptionkey@0"
I20260812 02:36:49.066015 14809 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502156971023-7698-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502156971023-7698-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502156971023-7698-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:36:49.091280 14809 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:36:49.096982 14809 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:36:49.097342 14809 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:36:49.098563 14809 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:36:49.100061 14809 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:36:49.100147 14809 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:36:49.100237 14809 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:36:49.100282 14809 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:36:49.134291 14809 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.130:36645
I20260812 02:36:49.134343 14955 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.130:36645 every 8 connection(s)
I20260812 02:36:49.135663 14809 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502156971023-7698-0/minicluster-data/ts-1/data/info.pb
I20260812 02:36:49.139889 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 14809
I20260812 02:36:49.140084 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502156971023-7698-0/minicluster-data/ts-1/wal/instance
I20260812 02:36:49.140699 7698 external_mini_cluster.cc:1428] Setting key 73b1ca7790aa164a7d6ec447854cfa48
Aug 12 02:36:49 dist-test-slave-mn4r krb5kdc[14512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502209, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.130@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 02:36:49.160151 14959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:49.138309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:56381 (local address 127.7.132.190:34501)
0812 02:36:49.138809 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:49.138814 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:49.139881 (+ 1067us) server_negotiation.cc:407] Connection header received
0812 02:36:49.142377 (+ 2496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:49.142382 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:49.142450 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:49.142542 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:49.144634 (+ 2092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:49.145333 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:49.147867 (+ 2534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:49.148289 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:49.148388 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:49.152722 (+ 4334us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:49.152732 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:49.152740 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:49.152787 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:49.156573 (+ 3786us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:49.157351 (+ 778us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:49.157359 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:49.157365 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:49.157545 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:49.157982 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:49.157990 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:49.157996 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:49.158433 (+ 437us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:49.158607 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:49.159363 (+ 756us) server_negotiation.cc:299] Negotiation successful
0812 02:36:49.159647 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":386,"thread_start_us":279,"threads_started":1}
I20260812 02:36:49.160835 14960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:49.138782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:34501 (local address 127.7.132.130:56381)
0812 02:36:49.139636 (+ 854us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:49.139692 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:49.142030 (+ 2338us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:49.142812 (+ 782us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:49.142820 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:49.143533 (+ 713us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:36:49.144425 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:49.144456 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:49.145505 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:49.145513 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:49.147713 (+ 2200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:49.147733 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:49.148526 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:49.148536 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:49.148868 (+ 332us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:49.149659 (+ 791us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:36:49.149693 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:36:49.152458 (+ 2765us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:36:49.156727 (+ 4269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:49.156734 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:49.156748 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:49.157199 (+ 451us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:49.157665 (+ 466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:49.157668 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:49.157670 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:49.157856 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:49.158579 (+ 723us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:36:49.158584 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:36:49.158937 (+ 353us) client_negotiation.cc:769] Sending connection context
0812 02:36:49.159194 (+ 257us) client_negotiation.cc:240] Negotiation successful
0812 02:36:49.159615 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":503,"thread_start_us":289,"threads_started":1}
I20260812 02:36:49.162647 14956 heartbeater.cc:347] Connected to a master server at 127.7.132.190:34501
I20260812 02:36:49.162994 14956 heartbeater.cc:464] Registering TS with master...
I20260812 02:36:49.163791 14956 heartbeater.cc:511] Master 127.7.132.190:34501 requested a full tablet report, sending...
I20260812 02:36:49.165418 14566 ts_manager.cc:194] Registered new tserver with Master: a28fd210b5574adc98317f2ed99fa761 (127.7.132.130:36645)
I20260812 02:36:49.166787 14566 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.130@KRBTEST.COM'} at 127.7.132.130:56381
WARNING: no policy specified for kudu/127.7.132.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.131.keytab.
Entry for principal kudu/127.7.132.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.131.keytab.
I20260812 02:36:49.201159 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502156971023-7698-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502156971023-7698-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502156971023-7698-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502156971023-7698-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.131:0
--local_ip_for_outbound_sockets=127.7.132.131
--webserver_interface=127.7.132.131
--webserver_port=0
--tserver_master_addrs=127.7.132.190:34501
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.131.keytab
--principal=kudu/127.7.132.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:39719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:36:49.555428 14963 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:36:49.555723 14963 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:36:49.555823 14963 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:36:49.563006 14963 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:36:49.563122 14963 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:36:49.563235 14963 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.131
I20260812 02:36:49.573271 14963 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:39719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502156971023-7698-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502156971023-7698-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.131.keytab
--principal=kudu/127.7.132.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502156971023-7698-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.132.131
--webserver_port=0
--tserver_master_addrs=127.7.132.190:34501
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.131
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502156971023-7698-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:36:49.574649 14963 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:36:49.575942 14963 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:36:49.589735 14978 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:36:49.589706 14975 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:36:49.590715 14973 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:36:49.592525 14963 server_base.cc:1034] running on GCE node
I20260812 02:36:49.593179 14963 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:36:49.594228 14963 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:36:49.595429 14963 hybrid_clock.cc:648] HybridClock initialized: now 1786502209595381 us; error 43 us; skew 500 ppm
I20260812 02:36:49.595760 14963 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:36:49 dist-test-slave-mn4r krb5kdc[14512](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502209, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:36:49.599002 14963 init.cc:377] Logged in from keytab as kudu/127.7.132.131@KRBTEST.COM (short username kudu)
I20260812 02:36:49.600566 14963 webserver.cc:460] Webserver started at http://127.7.132.131:39051/ using document root <none> and password file <none>
I20260812 02:36:49.601313 14963 fs_manager.cc:359] Metadata directory not provided
I20260812 02:36:49.601450 14963 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:36:49.601815 14963 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:36:49.604966 14963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502156971023-7698-0/minicluster-data/ts-2/data/instance:
uuid: "01ae5526ea0e476fbb5489623ad998ca"
format_stamp: "Formatted at 2026-08-12 02:36:49 on dist-test-slave-mn4r"
server_key: "40b4b97b82c9cea971d8e3eb494bdb87"
server_key_iv: "87d581fd8af65adf91ad036847e10e0c"
server_key_version: "encryptionkey@0"
I20260812 02:36:49.605779 14963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502156971023-7698-0/minicluster-data/ts-2/wal/instance:
uuid: "01ae5526ea0e476fbb5489623ad998ca"
format_stamp: "Formatted at 2026-08-12 02:36:49 on dist-test-slave-mn4r"
server_key: "40b4b97b82c9cea971d8e3eb494bdb87"
server_key_iv: "87d581fd8af65adf91ad036847e10e0c"
server_key_version: "encryptionkey@0"
I20260812 02:36:49.611819 14963 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.003s
I20260812 02:36:49.615312 14987 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:36:49.617034 14963 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 02:36:49.617177 14963 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502156971023-7698-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502156971023-7698-0/minicluster-data/ts-2/wal
uuid: "01ae5526ea0e476fbb5489623ad998ca"
format_stamp: "Formatted at 2026-08-12 02:36:49 on dist-test-slave-mn4r"
server_key: "40b4b97b82c9cea971d8e3eb494bdb87"
server_key_iv: "87d581fd8af65adf91ad036847e10e0c"
server_key_version: "encryptionkey@0"
I20260812 02:36:49.617362 14963 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502156971023-7698-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502156971023-7698-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502156971023-7698-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:36:49.621006 14801 heartbeater.cc:503] Master 127.7.132.190:34501 was elected leader, sending a full tablet report...
I20260812 02:36:49.643396 14963 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:36:49.648245 14963 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:36:49.648578 14963 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:36:49.649636 14963 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:36:49.651182 14963 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:36:49.651262 14963 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:36:49.651441 14963 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:36:49.651494 14963 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:36:49.685010 14963 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.131:35517
I20260812 02:36:49.685058 15123 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.131:35517 every 8 connection(s)
I20260812 02:36:49.686446 14963 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502156971023-7698-0/minicluster-data/ts-2/data/info.pb
I20260812 02:36:49.688006 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 14963
I20260812 02:36:49.688199 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502156971023-7698-0/minicluster-data/ts-2/wal/instance
I20260812 02:36:49.688835 7698 external_mini_cluster.cc:1428] Setting key 6a9e9351a8e3e4835bf2c9c16361f1ad
Aug 12 02:36:49 dist-test-slave-mn4r krb5kdc[14512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502209, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.131@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 02:36:49.709146 15126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:49.689519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:41049 (local address 127.7.132.190:34501)
0812 02:36:49.690315 (+ 796us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:49.690320 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:49.691303 (+ 983us) server_negotiation.cc:407] Connection header received
0812 02:36:49.693415 (+ 2112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:49.693419 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:49.693484 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:49.693577 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:49.695195 (+ 1618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:49.696059 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:49.697647 (+ 1588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:49.698065 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:49.698242 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:49.702133 (+ 3891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:49.702142 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:49.702147 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:49.702196 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:49.705727 (+ 3531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:49.706782 (+ 1055us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:49.706786 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:49.706789 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:49.706944 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:49.707404 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:49.707408 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:49.707410 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:49.707704 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:49.707831 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:49.708683 (+ 852us) server_negotiation.cc:299] Negotiation successful
0812 02:36:49.708856 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":650,"thread_start_us":526,"threads_started":1}
I20260812 02:36:49.710327 15128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:49.690220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:34501 (local address 127.7.132.131:41049)
0812 02:36:49.691092 (+ 872us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:49.691166 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:49.693150 (+ 1984us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:49.693802 (+ 652us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:49.693811 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:49.694428 (+ 617us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:36:49.695011 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:49.695030 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:49.696221 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:49.696225 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:49.697500 (+ 1275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:49.697512 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:49.698339 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:49.698344 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:49.698545 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:49.699276 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:36:49.699298 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:36:49.701810 (+ 2512us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:36:49.705885 (+ 4075us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:49.705893 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:49.705907 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:49.706644 (+ 737us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:49.707052 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:49.707055 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:49.707058 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:49.707265 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:49.707870 (+ 605us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:36:49.707878 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:36:49.708281 (+ 403us) client_negotiation.cc:769] Sending connection context
0812 02:36:49.708562 (+ 281us) client_negotiation.cc:240] Negotiation successful
0812 02:36:49.709013 (+ 451us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":602,"thread_start_us":333,"threads_started":1}
I20260812 02:36:49.711880 15124 heartbeater.cc:347] Connected to a master server at 127.7.132.190:34501
I20260812 02:36:49.712222 15124 heartbeater.cc:464] Registering TS with master...
I20260812 02:36:49.712944 15124 heartbeater.cc:511] Master 127.7.132.190:34501 requested a full tablet report, sending...
I20260812 02:36:49.714416 14566 ts_manager.cc:194] Registered new tserver with Master: 01ae5526ea0e476fbb5489623ad998ca (127.7.132.131:35517)
I20260812 02:36:49.715265 14566 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.131@KRBTEST.COM'} at 127.7.132.131:41049
I20260812 02:36:49.717618 7698 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502156971023-7698-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502156971023-7698-0/krb5kdc/test-admin.keytab.
Aug 12 02:36:49 dist-test-slave-mn4r krb5kdc[14512](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502209, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:36:49.749425 7698 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260812 02:36:49.755352 15135 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:36:49.752486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51772 (local address 127.7.132.129:42989)
0812 02:36:49.753235 (+ 749us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:49.753243 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:49.753269 (+ 26us) server_negotiation.cc:407] Connection header received
0812 02:36:49.753333 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:49.753339 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:49.753564 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:49.753754 (+ 190us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:49.754880 (+ 1126us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51772: BlockingRecv error: recv got EOF from 127.0.0.1:51772 (error 108)
Metrics: {"server-negotiator.queue_time_us":556,"thread_start_us":381,"threads_started":1}
W20260812 02:36:49.755651 7698 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502156971023-7698-0/krb5kdc/krb5cc' of type FILE
Aug 12 02:36:49 dist-test-slave-mn4r krb5kdc[14512](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502209, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:36:49.757499 7698 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 12 02:36:49 dist-test-slave-mn4r krb5kdc[14512](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502209, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:36:49.759227 7698 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 12 02:36:49 dist-test-slave-mn4r krb5kdc[14512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502209, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.132.129@KRBTEST.COM
I20260812 02:36:49.776012 15135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:49.760817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51776 (local address 127.7.132.129:42989)
0812 02:36:49.761038 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:49.761044 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:49.761247 (+ 203us) server_negotiation.cc:407] Connection header received
0812 02:36:49.761494 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:49.761499 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:49.761579 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:49.761677 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:49.763043 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:49.764161 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:49.765442 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:49.766033 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:49.766226 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:49.770153 (+ 3927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:49.770171 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:49.770191 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:49.770236 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:49.773083 (+ 2847us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:49.773790 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:49.773806 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:49.773809 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:49.773933 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:49.774361 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:49.774365 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:49.774367 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:49.774755 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:49.775067 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:49.775440 (+ 373us) server_negotiation.cc:299] Negotiation successful
0812 02:36:49.775724 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
W20260812 02:36:49.782281 15135 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:36:49.780384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51786 (local address 127.7.132.129:42989)
0812 02:36:49.780580 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:49.780584 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:49.780930 (+ 346us) server_negotiation.cc:407] Connection header received
0812 02:36:49.781032 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:49.781036 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:49.781095 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:49.781181 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:49.781944 (+ 763us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51786: BlockingRecv error: recv got EOF from 127.0.0.1:51786 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
W20260812 02:36:49.782490 7698 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502156971023-7698-0/krb5kdc/krb5cc' of type FILE
Aug 12 02:36:49 dist-test-slave-mn4r krb5kdc[14512](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502209, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:36:49.784215 7698 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 12 02:36:49 dist-test-slave-mn4r krb5kdc[14512](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502209, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:36:49.785648 7698 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 12 02:36:49 dist-test-slave-mn4r krb5kdc[14512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502209, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.132.129@KRBTEST.COM
I20260812 02:36:49.801083 15135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:49.787210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51790 (local address 127.7.132.129:42989)
0812 02:36:49.787463 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:49.787469 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:49.787741 (+ 272us) server_negotiation.cc:407] Connection header received
0812 02:36:49.787936 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:49.787941 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:49.788021 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:49.788130 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:49.789139 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:49.789863 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:49.791152 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:49.791595 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:49.791731 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:49.795919 (+ 4188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:49.795926 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:49.795931 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:49.795978 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:49.798581 (+ 2603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:49.799204 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:49.799208 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:49.799211 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:49.799334 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:49.799690 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:49.799693 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:49.799695 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:49.799955 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:49.800162 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:49.800407 (+ 245us) server_negotiation.cc:299] Negotiation successful
0812 02:36:49.800725 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
W20260812 02:36:49.806056 15135 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:36:49.803948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51798 (local address 127.7.132.129:42989)
0812 02:36:49.804188 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:49.804197 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:49.804511 (+ 314us) server_negotiation.cc:407] Connection header received
0812 02:36:49.804717 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:49.804724 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:49.804810 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:49.804984 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:49.805778 (+ 794us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51798: BlockingRecv error: recv got EOF from 127.0.0.1:51798 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
W20260812 02:36:49.806293 7698 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786502156971023-7698-0/krb5kdc/krb5cc' of type FILE
Aug 12 02:36:49 dist-test-slave-mn4r krb5kdc[14512](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502209, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:36:49.807871 7698 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 12 02:36:49 dist-test-slave-mn4r krb5kdc[14512](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502209, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:36:49.809139 7698 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 12 02:36:49 dist-test-slave-mn4r krb5kdc[14512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502209, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.132.129@KRBTEST.COM
I20260812 02:36:49.824463 15135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:49.810415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51812 (local address 127.7.132.129:42989)
0812 02:36:49.810625 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:49.810634 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:49.810973 (+ 339us) server_negotiation.cc:407] Connection header received
0812 02:36:49.811089 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:49.811093 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:49.811147 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:49.811226 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:49.812223 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:49.812926 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:49.814164 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:49.814571 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:49.814663 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:49.818505 (+ 3842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:49.818510 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:49.818513 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:49.818549 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:49.821274 (+ 2725us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:49.822199 (+ 925us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:49.822203 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:49.822205 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:49.822368 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:49.822861 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:49.822866 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:49.822868 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:49.823264 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:49.823542 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:49.823859 (+ 317us) server_negotiation.cc:299] Negotiation successful
0812 02:36:49.824098 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 02:36:49.826261 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 14639
I20260812 02:36:49.838587 15135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:49.827019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51828 (local address 127.7.132.129:42989)
0812 02:36:49.827185 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:49.827189 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:49.827563 (+ 374us) server_negotiation.cc:407] Connection header received
0812 02:36:49.827682 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:49.827686 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:49.827744 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:49.827821 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:49.828830 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:49.829603 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:49.830862 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:49.831295 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:49.831399 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:49.833369 (+ 1970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:49.833374 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:49.833377 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:49.833408 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:49.835858 (+ 2450us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:49.836538 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:49.836545 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:49.836548 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:49.836749 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:49.837168 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:49.837173 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:49.837175 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:49.837489 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:49.837728 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:49.838022 (+ 294us) server_negotiation.cc:299] Negotiation successful
0812 02:36:49.838288 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 02:36:49.839229 14794 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:36:49.924719 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 14639
I20260812 02:36:49.946854 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 14809
Aug 12 02:36:49 dist-test-slave-mn4r krb5kdc[14512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502209, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.132.130@KRBTEST.COM
I20260812 02:36:49.962802 15154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:49.947810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35232 (local address 127.7.132.130:36645)
0812 02:36:49.948474 (+ 664us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:49.948487 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:49.948516 (+ 29us) server_negotiation.cc:407] Connection header received
0812 02:36:49.948580 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:49.948586 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:49.948852 (+ 266us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:49.948989 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:49.949921 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:49.951033 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:49.952280 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:49.952841 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:49.952939 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:49.956799 (+ 3860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:49.956811 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:49.956824 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:49.956866 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:49.959782 (+ 2916us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:49.960575 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:49.960582 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:49.960585 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:49.960709 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:49.961127 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:49.961130 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:49.961132 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:49.961462 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:49.961701 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:49.962068 (+ 367us) server_negotiation.cc:299] Negotiation successful
0812 02:36:49.962476 (+ 408us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":512,"thread_start_us":373,"threads_started":1}
I20260812 02:36:49.964087 14951 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:36:50.047263 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 14809
I20260812 02:36:50.068672 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 14963
Aug 12 02:36:50 dist-test-slave-mn4r krb5kdc[14512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502209, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.132.131@KRBTEST.COM
I20260812 02:36:50.086346 15156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:50.069948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52954 (local address 127.7.132.131:35517)
0812 02:36:50.070678 (+ 730us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:50.070685 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:50.070718 (+ 33us) server_negotiation.cc:407] Connection header received
0812 02:36:50.070782 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:50.070789 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:50.071021 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:50.071217 (+ 196us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:50.072435 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:50.073490 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:50.074994 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:50.075604 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:50.075770 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:50.080126 (+ 4356us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:50.080142 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:50.080158 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:50.080199 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:50.083212 (+ 3013us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:50.084051 (+ 839us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:50.084059 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:50.084061 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:50.084198 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:50.084617 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:50.084622 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:50.084641 (+ 19us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:50.085019 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:50.085371 (+ 352us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:50.085669 (+ 298us) server_negotiation.cc:299] Negotiation successful
0812 02:36:50.085966 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":568,"thread_start_us":402,"threads_started":1}
I20260812 02:36:50.087754 15116 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:36:50.171561 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 14963
I20260812 02:36:50.191649 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 14529
I20260812 02:36:50.192905 14602 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:36:50.297830 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 14529
2026-08-12T02:36:50Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (4256 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 02:36:50 dist-test-slave-mn4r krb5kdc[15163](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 02:36:50 dist-test-slave-mn4r krb5kdc[15163](info): set up 2 sockets
Aug 12 02:36:50 dist-test-slave-mn4r krb5kdc[15163](info): commencing operation
krb5kdc: starting...
W20260812 02:36:51.515149 7698 mini_kdc.cc:121] Time spent starting KDC: real 1.169s user 0.003s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/krb5kdc/test-user.keytab.
Aug 12 02:36:51 dist-test-slave-mn4r krb5kdc[15163](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502211, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T02:36:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T02:36:51Z Disabled control of system clock
WARNING: no policy specified for oryx/127.7.132.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.132.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.132.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/krb5kdc/oryx_127.7.132.190.keytab.
Entry for principal oryx/127.7.132.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/krb5kdc/oryx_127.7.132.190.keytab.
I20260812 02:36:51.766003 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:33723
--webserver_interface=127.7.132.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/krb5kdc/oryx_127.7.132.190.keytab
--principal=oryx/127.7.132.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:43909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.132.190:33723
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:36:52.106204 15180 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:36:52.106503 15180 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:36:52.106683 15180 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:36:52.113982 15180 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:36:52.114107 15180 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:36:52.114212 15180 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:36:52.114321 15180 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:36:52.114403 15180 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:36:52.124491 15180 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:43909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.132.190:33723
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/krb5kdc/oryx_127.7.132.190.keytab
--principal=oryx/127.7.132.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:33723
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.132.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:36:52.125912 15180 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:36:52.127408 15180 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:36:52.139722 15188 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:36:52.139748 15185 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:36:52.140486 15191 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:36:52.140966 15180 server_base.cc:1034] running on GCE node
I20260812 02:36:52.141844 15180 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:36:52.143445 15180 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:36:52.144730 15180 hybrid_clock.cc:648] HybridClock initialized: now 1786502212144677 us; error 53 us; skew 500 ppm
I20260812 02:36:52.145131 15180 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:36:52 dist-test-slave-mn4r krb5kdc[15163](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502212, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:36:52.148469 15180 init.cc:377] Logged in from keytab as oryx/127.7.132.190@KRBTEST.COM (short username oryx)
I20260812 02:36:52.150137 15180 webserver.cc:460] Webserver started at http://127.7.132.190:40631/ using document root <none> and password file <none>
I20260812 02:36:52.150871 15180 fs_manager.cc:359] Metadata directory not provided
I20260812 02:36:52.151029 15180 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:36:52.151360 15180 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:36:52.154546 15180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/master-0/data/instance:
uuid: "82e54874d347460c85c36da26b3bb8c8"
format_stamp: "Formatted at 2026-08-12 02:36:52 on dist-test-slave-mn4r"
server_key: "144f75e58f9dfe497cf3201cd93ca7cb"
server_key_iv: "4de778d91ba0e749d2309b57c39de631"
server_key_version: "encryptionkey@0"
I20260812 02:36:52.155408 15180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/master-0/wal/instance:
uuid: "82e54874d347460c85c36da26b3bb8c8"
format_stamp: "Formatted at 2026-08-12 02:36:52 on dist-test-slave-mn4r"
server_key: "144f75e58f9dfe497cf3201cd93ca7cb"
server_key_iv: "4de778d91ba0e749d2309b57c39de631"
server_key_version: "encryptionkey@0"
I20260812 02:36:52.160794 15180 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260812 02:36:52.164433 15204 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:36:52.165781 15180 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260812 02:36:52.166014 15180 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/master-0/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/master-0/wal
uuid: "82e54874d347460c85c36da26b3bb8c8"
format_stamp: "Formatted at 2026-08-12 02:36:52 on dist-test-slave-mn4r"
server_key: "144f75e58f9dfe497cf3201cd93ca7cb"
server_key_iv: "4de778d91ba0e749d2309b57c39de631"
server_key_version: "encryptionkey@0"
I20260812 02:36:52.166293 15180 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:36:52.184777 15180 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:36:52.189124 15180 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:36:52.189435 15180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:36:52.208293 15180 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.190:33723
I20260812 02:36:52.208324 15264 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.190:33723 every 8 connection(s)
I20260812 02:36:52.209734 15180 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
I20260812 02:36:52.213552 15265 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:36:52.220229 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 15180
I20260812 02:36:52.220463 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/master-0/wal/instance
I20260812 02:36:52.221330 7698 external_mini_cluster.cc:1428] Setting key 3e655fcfa5b7d46356d90a36f3168de1
I20260812 02:36:52.225157 15265 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82e54874d347460c85c36da26b3bb8c8: Bootstrap starting.
I20260812 02:36:52.228725 15265 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 82e54874d347460c85c36da26b3bb8c8: Neither blocks nor log segments found. Creating new log.
I20260812 02:36:52.229833 15265 log.cc:824] T 00000000000000000000000000000000 P 82e54874d347460c85c36da26b3bb8c8: Log is configured to *not* fsync() on all Append() calls
I20260812 02:36:52.232735 15265 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82e54874d347460c85c36da26b3bb8c8: No bootstrap required, opened a new log
Aug 12 02:36:52 dist-test-slave-mn4r krb5kdc[15163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502211, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM
I20260812 02:36:52.242825 15265 raft_consensus.cc:348] T 00000000000000000000000000000000 P 82e54874d347460c85c36da26b3bb8c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82e54874d347460c85c36da26b3bb8c8" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 33723 } }
I20260812 02:36:52.243364 15265 raft_consensus.cc:374] T 00000000000000000000000000000000 P 82e54874d347460c85c36da26b3bb8c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:36:52.243485 15265 raft_consensus.cc:729] T 00000000000000000000000000000000 P 82e54874d347460c85c36da26b3bb8c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82e54874d347460c85c36da26b3bb8c8, State: Initialized, Role: FOLLOWER
I20260812 02:36:52.244292 15265 consensus_queue.cc:260] T 00000000000000000000000000000000 P 82e54874d347460c85c36da26b3bb8c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82e54874d347460c85c36da26b3bb8c8" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 33723 } }
I20260812 02:36:52.244583 15265 raft_consensus.cc:388] T 00000000000000000000000000000000 P 82e54874d347460c85c36da26b3bb8c8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 02:36:52.244736 15265 raft_consensus.cc:482] T 00000000000000000000000000000000 P 82e54874d347460c85c36da26b3bb8c8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 02:36:52.244927 15265 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 82e54874d347460c85c36da26b3bb8c8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:36:52.246135 15273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:52.223549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42154 (local address 127.7.132.190:33723)
0812 02:36:52.224334 (+ 785us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:52.224347 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:52.224389 (+ 42us) server_negotiation.cc:407] Connection header received
0812 02:36:52.226225 (+ 1836us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:52.226249 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:52.226636 (+ 387us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:52.227064 (+ 428us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:52.228661 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:52.230130 (+ 1469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:52.231426 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:52.232108 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:52.232313 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:52.236454 (+ 4141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:52.236467 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:52.236488 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:52.236540 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:52.240787 (+ 4247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:52.241522 (+ 735us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:52.241531 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:52.241542 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:52.241766 (+ 224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:52.242289 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:52.242294 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:52.242297 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:52.243098 (+ 801us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:52.243453 (+ 355us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:52.244061 (+ 608us) server_negotiation.cc:299] Negotiation successful
0812 02:36:52.244554 (+ 493us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":582,"thread_start_us":381,"threads_started":1}
I20260812 02:36:52.247788 15265 raft_consensus.cc:504] T 00000000000000000000000000000000 P 82e54874d347460c85c36da26b3bb8c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82e54874d347460c85c36da26b3bb8c8" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 33723 } }
I20260812 02:36:52.248314 15265 leader_election.cc:302] T 00000000000000000000000000000000 P 82e54874d347460c85c36da26b3bb8c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 82e54874d347460c85c36da26b3bb8c8; no voters:
I20260812 02:36:52.249207 15265 leader_election.cc:288] T 00000000000000000000000000000000 P 82e54874d347460c85c36da26b3bb8c8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 02:36:52.249295 15276 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 82e54874d347460c85c36da26b3bb8c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:36:52.250159 15276 raft_consensus.cc:686] T 00000000000000000000000000000000 P 82e54874d347460c85c36da26b3bb8c8 [term 1 LEADER]: Becoming Leader. State: Replica: 82e54874d347460c85c36da26b3bb8c8, State: Running, Role: LEADER
I20260812 02:36:52.250753 15276 consensus_queue.cc:237] T 00000000000000000000000000000000 P 82e54874d347460c85c36da26b3bb8c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82e54874d347460c85c36da26b3bb8c8" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 33723 } }
I20260812 02:36:52.251497 15265 sys_catalog.cc:564] T 00000000000000000000000000000000 P 82e54874d347460c85c36da26b3bb8c8 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:36:52.257745 15279 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82e54874d347460c85c36da26b3bb8c8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "82e54874d347460c85c36da26b3bb8c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82e54874d347460c85c36da26b3bb8c8" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 33723 } } }
I20260812 02:36:52.258204 15279 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82e54874d347460c85c36da26b3bb8c8 [sys.catalog]: This master's current role is: LEADER
I20260812 02:36:52.258782 15280 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82e54874d347460c85c36da26b3bb8c8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 82e54874d347460c85c36da26b3bb8c8. Latest consensus state: current_term: 1 leader_uuid: "82e54874d347460c85c36da26b3bb8c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82e54874d347460c85c36da26b3bb8c8" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 33723 } } }
I20260812 02:36:52.259109 15295 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:36:52.259111 15280 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82e54874d347460c85c36da26b3bb8c8 [sys.catalog]: This master's current role is: LEADER
I20260812 02:36:52.272816 15295 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 02:36:52.282660 15295 catalog_manager.cc:1347] Generated new cluster ID: eb5dad367dc04d1ba16a1cfb4c2dc8a7
I20260812 02:36:52.282797 15295 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 02:36:52.300594 15295 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 02:36:52.301949 15295 catalog_manager.cc:1504] Loading token signing keys...
I20260812 02:36:52.311106 15295 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 82e54874d347460c85c36da26b3bb8c8: Generated new TSK 0
I20260812 02:36:52.311995 15295 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.7.132.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.132.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.132.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/krb5kdc/oryx_127.7.132.129.keytab.
Entry for principal oryx/127.7.132.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/krb5kdc/oryx_127.7.132.129.keytab.
I20260812 02:36:52.389097 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.129:0
--local_ip_for_outbound_sockets=127.7.132.129
--webserver_interface=127.7.132.129
--webserver_port=0
--tserver_master_addrs=127.7.132.190:33723
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/krb5kdc/oryx_127.7.132.129.keytab
--principal=oryx/127.7.132.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:43909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:36:52.736719 15305 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:36:52.737012 15305 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:36:52.737164 15305 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:36:52.744594 15305 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:36:52.744756 15305 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:36:52.744997 15305 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.129
I20260812 02:36:52.755281 15305 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:43909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/krb5kdc/oryx_127.7.132.129.keytab
--principal=oryx/127.7.132.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.132.129
--webserver_port=0
--tserver_master_addrs=127.7.132.190:33723
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.129
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:36:52.756830 15305 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:36:52.758291 15305 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:36:52.772063 15316 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:36:52.772112 15313 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:36:52.772367 15312 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:36:52.779762 15305 server_base.cc:1034] running on GCE node
I20260812 02:36:52.780401 15305 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:36:52.781349 15305 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:36:52.782588 15305 hybrid_clock.cc:648] HybridClock initialized: now 1786502212782538 us; error 66 us; skew 500 ppm
I20260812 02:36:52.782924 15305 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:36:52 dist-test-slave-mn4r krb5kdc[15163](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502212, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.132.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:36:52.793437 15305 init.cc:377] Logged in from keytab as oryx/127.7.132.129@KRBTEST.COM (short username oryx)
I20260812 02:36:52.795146 15305 webserver.cc:460] Webserver started at http://127.7.132.129:44947/ using document root <none> and password file <none>
I20260812 02:36:52.795936 15305 fs_manager.cc:359] Metadata directory not provided
I20260812 02:36:52.796096 15305 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:36:52.796425 15305 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:36:52.799516 15305 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/ts-0/data/instance:
uuid: "ed2e8f19bfd642ac95e6b32d054ce8f4"
format_stamp: "Formatted at 2026-08-12 02:36:52 on dist-test-slave-mn4r"
server_key: "ac74a91345f24659f567325fd15c4afd"
server_key_iv: "bd41d6e3f396cbbdbd2b95b8d68ee332"
server_key_version: "encryptionkey@0"
I20260812 02:36:52.800309 15305 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/ts-0/wal/instance:
uuid: "ed2e8f19bfd642ac95e6b32d054ce8f4"
format_stamp: "Formatted at 2026-08-12 02:36:52 on dist-test-slave-mn4r"
server_key: "ac74a91345f24659f567325fd15c4afd"
server_key_iv: "bd41d6e3f396cbbdbd2b95b8d68ee332"
server_key_version: "encryptionkey@0"
I20260812 02:36:52.805603 15305 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260812 02:36:52.809069 15324 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:36:52.810532 15305 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 02:36:52.810716 15305 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/ts-0/wal
uuid: "ed2e8f19bfd642ac95e6b32d054ce8f4"
format_stamp: "Formatted at 2026-08-12 02:36:52 on dist-test-slave-mn4r"
server_key: "ac74a91345f24659f567325fd15c4afd"
server_key_iv: "bd41d6e3f396cbbdbd2b95b8d68ee332"
server_key_version: "encryptionkey@0"
I20260812 02:36:52.810936 15305 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:36:52.824492 15305 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:36:52.829116 15305 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:36:52.829465 15305 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:36:52.830538 15305 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:36:52.832036 15305 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:36:52.832150 15305 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:36:52.832322 15305 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:36:52.832407 15305 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:36:52.868852 15305 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.129:34169
I20260812 02:36:52.868888 15458 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.129:34169 every 8 connection(s)
I20260812 02:36:52.870620 15305 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/ts-0/data/info.pb
I20260812 02:36:52.877971 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 15305
I20260812 02:36:52.878247 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/ts-0/wal/instance
I20260812 02:36:52.879123 7698 external_mini_cluster.cc:1428] Setting key 865e83396fd86c73df4d1875fb7660d7
Aug 12 02:36:52 dist-test-slave-mn4r krb5kdc[15163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502212, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.132.129@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM
I20260812 02:36:52.895337 15462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:52.873565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:38775 (local address 127.7.132.190:33723)
0812 02:36:52.874006 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:52.874011 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:52.875107 (+ 1096us) server_negotiation.cc:407] Connection header received
0812 02:36:52.877003 (+ 1896us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:52.877009 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:52.877113 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:52.877232 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:52.879571 (+ 2339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:52.880465 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:52.882828 (+ 2363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:52.883309 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:52.883417 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:52.888258 (+ 4841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:52.888265 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:52.888269 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:52.888308 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:52.891266 (+ 2958us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:52.892347 (+ 1081us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:52.892357 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:52.892364 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:52.892583 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:52.893042 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:52.893051 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:52.893057 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:52.893498 (+ 441us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:52.893667 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:52.894483 (+ 816us) server_negotiation.cc:299] Negotiation successful
0812 02:36:52.894789 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":227,"threads_started":1}
I20260812 02:36:52.895936 15463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:52.874216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:33723 (local address 127.7.132.129:38775)
0812 02:36:52.874929 (+ 713us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:52.874977 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:52.876714 (+ 1737us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:52.877545 (+ 831us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:52.877557 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:52.878440 (+ 883us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:36:52.879359 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:52.879387 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:52.880633 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:52.880639 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:52.882626 (+ 1987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:52.882646 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:52.883562 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:52.883573 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:52.883919 (+ 346us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:52.884754 (+ 835us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:36:52.884788 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:36:52.887977 (+ 3189us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:36:52.891448 (+ 3471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:52.891461 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:52.891483 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:52.892165 (+ 682us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:52.892715 (+ 550us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:52.892718 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:52.892720 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:52.892917 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:52.893634 (+ 717us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:36:52.893640 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:36:52.893983 (+ 343us) client_negotiation.cc:769] Sending connection context
0812 02:36:52.894292 (+ 309us) client_negotiation.cc:240] Negotiation successful
0812 02:36:52.894731 (+ 439us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":276,"threads_started":1}
I20260812 02:36:52.897603 15459 heartbeater.cc:347] Connected to a master server at 127.7.132.190:33723
I20260812 02:36:52.897938 15459 heartbeater.cc:464] Registering TS with master...
I20260812 02:36:52.898734 15459 heartbeater.cc:511] Master 127.7.132.190:33723 requested a full tablet report, sending...
I20260812 02:36:52.901021 15223 ts_manager.cc:194] Registered new tserver with Master: ed2e8f19bfd642ac95e6b32d054ce8f4 (127.7.132.129:34169)
I20260812 02:36:52.902621 15223 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.132.129@KRBTEST.COM'} at 127.7.132.129:38775
WARNING: no policy specified for oryx/127.7.132.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.132.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.132.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/krb5kdc/oryx_127.7.132.130.keytab.
Entry for principal oryx/127.7.132.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/krb5kdc/oryx_127.7.132.130.keytab.
I20260812 02:36:52.941603 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.130:0
--local_ip_for_outbound_sockets=127.7.132.130
--webserver_interface=127.7.132.130
--webserver_port=0
--tserver_master_addrs=127.7.132.190:33723
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/krb5kdc/oryx_127.7.132.130.keytab
--principal=oryx/127.7.132.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:43909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:36:53.287076 15466 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:36:53.287403 15466 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:36:53.287583 15466 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:36:53.294914 15466 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:36:53.295070 15466 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:36:53.295240 15466 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.130
I20260812 02:36:53.305804 15466 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:43909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/krb5kdc/oryx_127.7.132.130.keytab
--principal=oryx/127.7.132.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.132.130
--webserver_port=0
--tserver_master_addrs=127.7.132.190:33723
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.130
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:36:53.307261 15466 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:36:53.308678 15466 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:36:53.323170 15473 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:36:53.323139 15475 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:36:53.323206 15472 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:36:53.327564 15466 server_base.cc:1034] running on GCE node
I20260812 02:36:53.328282 15466 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:36:53.329376 15466 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:36:53.330631 15466 hybrid_clock.cc:648] HybridClock initialized: now 1786502213330555 us; error 61 us; skew 500 ppm
I20260812 02:36:53.331024 15466 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:36:53 dist-test-slave-mn4r krb5kdc[15163](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502213, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.132.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:36:53.334396 15466 init.cc:377] Logged in from keytab as oryx/127.7.132.130@KRBTEST.COM (short username oryx)
I20260812 02:36:53.335953 15466 webserver.cc:460] Webserver started at http://127.7.132.130:38257/ using document root <none> and password file <none>
I20260812 02:36:53.336728 15466 fs_manager.cc:359] Metadata directory not provided
I20260812 02:36:53.336880 15466 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:36:53.337203 15466 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:36:53.340212 15466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/ts-1/data/instance:
uuid: "07b5778ae04246ec9f152a523a1ef33f"
format_stamp: "Formatted at 2026-08-12 02:36:53 on dist-test-slave-mn4r"
server_key: "7077c4798e2ada1d7fdc890ba8c31ee4"
server_key_iv: "ba9000397d73642e69786e6ded1f6a82"
server_key_version: "encryptionkey@0"
I20260812 02:36:53.340999 15466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/ts-1/wal/instance:
uuid: "07b5778ae04246ec9f152a523a1ef33f"
format_stamp: "Formatted at 2026-08-12 02:36:53 on dist-test-slave-mn4r"
server_key: "7077c4798e2ada1d7fdc890ba8c31ee4"
server_key_iv: "ba9000397d73642e69786e6ded1f6a82"
server_key_version: "encryptionkey@0"
I20260812 02:36:53.347862 15466 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.007s
I20260812 02:36:53.352079 15482 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:36:53.353514 15466 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 02:36:53.353689 15466 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/ts-1/wal
uuid: "07b5778ae04246ec9f152a523a1ef33f"
format_stamp: "Formatted at 2026-08-12 02:36:53 on dist-test-slave-mn4r"
server_key: "7077c4798e2ada1d7fdc890ba8c31ee4"
server_key_iv: "ba9000397d73642e69786e6ded1f6a82"
server_key_version: "encryptionkey@0"
I20260812 02:36:53.353899 15466 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:36:53.369645 15466 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:36:53.373724 15466 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:36:53.374023 15466 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:36:53.374995 15466 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:36:53.376375 15466 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:36:53.376487 15466 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:36:53.376652 15466 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:36:53.376725 15466 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:36:53.412086 15466 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.130:36973
I20260812 02:36:53.412177 15628 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.130:36973 every 8 connection(s)
I20260812 02:36:53.413446 15466 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/ts-1/data/info.pb
I20260812 02:36:53.417372 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 15466
I20260812 02:36:53.417554 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/ts-1/wal/instance
I20260812 02:36:53.418160 7698 external_mini_cluster.cc:1428] Setting key 5a5dee53a400f03755f6a32182e934ce
Aug 12 02:36:53 dist-test-slave-mn4r krb5kdc[15163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502213, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.132.130@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM
I20260812 02:36:53.435123 15631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:53.416225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:45109 (local address 127.7.132.190:33723)
0812 02:36:53.416669 (+ 444us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:53.416673 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:53.417827 (+ 1154us) server_negotiation.cc:407] Connection header received
0812 02:36:53.419810 (+ 1983us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:53.419816 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:53.419908 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:53.420039 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:53.421579 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:53.422418 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:53.424008 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:53.424449 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:53.424583 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:53.428534 (+ 3951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:53.428540 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:53.428543 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:53.428577 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:53.431599 (+ 3022us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:53.432606 (+ 1007us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:53.432610 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:53.432612 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:53.432739 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:53.433266 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:53.433269 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:53.433271 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:53.433530 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:53.433657 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:53.434637 (+ 980us) server_negotiation.cc:299] Negotiation successful
0812 02:36:53.434814 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":332,"thread_start_us":221,"threads_started":1}
I20260812 02:36:53.436962 15633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:53.416765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:33723 (local address 127.7.132.130:45109)
0812 02:36:53.417643 (+ 878us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:53.417706 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:53.419518 (+ 1812us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:53.420295 (+ 777us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:53.420304 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:53.420833 (+ 529us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:36:53.421400 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:53.421417 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:53.422593 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:53.422597 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:53.423850 (+ 1253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:53.423863 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:53.424699 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:53.424707 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:53.424947 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:53.425722 (+ 775us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:36:53.425746 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:36:53.428279 (+ 2533us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:36:53.431754 (+ 3475us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:53.431766 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:53.431786 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:53.432456 (+ 670us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:53.432867 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:53.432875 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:53.432881 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:53.433153 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:53.433685 (+ 532us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:36:53.433696 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:36:53.434221 (+ 525us) client_negotiation.cc:769] Sending connection context
0812 02:36:53.434582 (+ 361us) client_negotiation.cc:240] Negotiation successful
0812 02:36:53.435287 (+ 705us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":554,"thread_start_us":318,"threads_started":1}
I20260812 02:36:53.438671 15629 heartbeater.cc:347] Connected to a master server at 127.7.132.190:33723
I20260812 02:36:53.438993 15629 heartbeater.cc:464] Registering TS with master...
I20260812 02:36:53.439764 15629 heartbeater.cc:511] Master 127.7.132.190:33723 requested a full tablet report, sending...
I20260812 02:36:53.441561 15223 ts_manager.cc:194] Registered new tserver with Master: 07b5778ae04246ec9f152a523a1ef33f (127.7.132.130:36973)
I20260812 02:36:53.443065 15223 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.132.130@KRBTEST.COM'} at 127.7.132.130:45109
WARNING: no policy specified for oryx/127.7.132.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.132.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.132.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/krb5kdc/oryx_127.7.132.131.keytab.
Entry for principal oryx/127.7.132.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/krb5kdc/oryx_127.7.132.131.keytab.
I20260812 02:36:53.478821 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.131:0
--local_ip_for_outbound_sockets=127.7.132.131
--webserver_interface=127.7.132.131
--webserver_port=0
--tserver_master_addrs=127.7.132.190:33723
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/krb5kdc/oryx_127.7.132.131.keytab
--principal=oryx/127.7.132.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:43909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:36:53.818195 15639 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:36:53.818480 15639 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:36:53.818634 15639 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:36:53.825929 15639 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:36:53.826040 15639 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:36:53.826265 15639 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.131
I20260812 02:36:53.836241 15639 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:43909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/krb5kdc/oryx_127.7.132.131.keytab
--principal=oryx/127.7.132.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.132.131
--webserver_port=0
--tserver_master_addrs=127.7.132.190:33723
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.131
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:36:53.837574 15639 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:36:53.838997 15639 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:36:53.852666 15646 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:36:53.852689 15649 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:36:53.852739 15647 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:36:53.854563 15639 server_base.cc:1034] running on GCE node
I20260812 02:36:53.855235 15639 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:36:53.856292 15639 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:36:53.857507 15639 hybrid_clock.cc:648] HybridClock initialized: now 1786502213857466 us; error 38 us; skew 500 ppm
I20260812 02:36:53.857846 15639 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:36:53 dist-test-slave-mn4r krb5kdc[15163](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502213, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.132.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:36:53.861173 15639 init.cc:377] Logged in from keytab as oryx/127.7.132.131@KRBTEST.COM (short username oryx)
I20260812 02:36:53.862649 15639 webserver.cc:460] Webserver started at http://127.7.132.131:45735/ using document root <none> and password file <none>
I20260812 02:36:53.863327 15639 fs_manager.cc:359] Metadata directory not provided
I20260812 02:36:53.863427 15639 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:36:53.863683 15639 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:36:53.866600 15639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/ts-2/data/instance:
uuid: "6c404a3267bc4a11a6850d91dec7b21d"
format_stamp: "Formatted at 2026-08-12 02:36:53 on dist-test-slave-mn4r"
server_key: "7de24234a859286210c3aca800c86795"
server_key_iv: "605bf975428e615b167ec11ea392d64c"
server_key_version: "encryptionkey@0"
I20260812 02:36:53.867311 15639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/ts-2/wal/instance:
uuid: "6c404a3267bc4a11a6850d91dec7b21d"
format_stamp: "Formatted at 2026-08-12 02:36:53 on dist-test-slave-mn4r"
server_key: "7de24234a859286210c3aca800c86795"
server_key_iv: "605bf975428e615b167ec11ea392d64c"
server_key_version: "encryptionkey@0"
I20260812 02:36:53.873070 15639 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260812 02:36:53.876487 15656 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:36:53.877823 15639 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260812 02:36:53.877997 15639 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/ts-2/wal
uuid: "6c404a3267bc4a11a6850d91dec7b21d"
format_stamp: "Formatted at 2026-08-12 02:36:53 on dist-test-slave-mn4r"
server_key: "7de24234a859286210c3aca800c86795"
server_key_iv: "605bf975428e615b167ec11ea392d64c"
server_key_version: "encryptionkey@0"
I20260812 02:36:53.878263 15639 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:36:53.898006 15639 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:36:53.902007 15639 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:36:53.902346 15639 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:36:53.903347 15639 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:36:53.904711 15639 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:36:53.904785 15639 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:36:53.904969 15639 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:36:53.905023 15639 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:36:53.905716 15459 heartbeater.cc:503] Master 127.7.132.190:33723 was elected leader, sending a full tablet report...
I20260812 02:36:53.940488 15639 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.131:35367
I20260812 02:36:53.940567 15801 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.131:35367 every 8 connection(s)
I20260812 02:36:53.941831 15639 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/ts-2/data/info.pb
I20260812 02:36:53.943563 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 15639
I20260812 02:36:53.943739 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/ts-2/wal/instance
I20260812 02:36:53.944312 7698 external_mini_cluster.cc:1428] Setting key 57c8681e827302483ae986822ae24dbf
Aug 12 02:36:53 dist-test-slave-mn4r krb5kdc[15163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502213, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.132.131@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM
I20260812 02:36:53.963268 15804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:53.944926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:60725 (local address 127.7.132.190:33723)
0812 02:36:53.945503 (+ 577us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:53.945510 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:53.946690 (+ 1180us) server_negotiation.cc:407] Connection header received
0812 02:36:53.948758 (+ 2068us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:53.948762 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:53.948842 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:53.948937 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:53.950527 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:53.951349 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:53.953181 (+ 1832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:53.953559 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:53.953657 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:53.957216 (+ 3559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:53.957221 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:53.957224 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:53.957258 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:53.960428 (+ 3170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:53.961117 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:53.961121 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:53.961123 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:53.961250 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:53.961647 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:53.961650 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:53.961652 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:53.961937 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:53.962057 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:53.962798 (+ 741us) server_negotiation.cc:299] Negotiation successful
0812 02:36:53.962973 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":403,"thread_start_us":260,"threads_started":1}
I20260812 02:36:53.964236 15805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:53.945606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:33723 (local address 127.7.132.131:60725)
0812 02:36:53.946508 (+ 902us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:53.946567 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:53.948514 (+ 1947us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:53.949181 (+ 667us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:53.949189 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:53.949700 (+ 511us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:36:53.950362 (+ 662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:53.950383 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:53.951553 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:53.951559 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:53.953051 (+ 1492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:53.953064 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:53.953744 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:53.953748 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:53.953943 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:53.954654 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:36:53.954676 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:36:53.957014 (+ 2338us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:36:53.960551 (+ 3537us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:53.960558 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:53.960570 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:53.961006 (+ 436us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:53.961343 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:53.961347 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:53.961349 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:53.961545 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:53.962052 (+ 507us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:36:53.962058 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:36:53.962426 (+ 368us) client_negotiation.cc:769] Sending connection context
0812 02:36:53.962685 (+ 259us) client_negotiation.cc:240] Negotiation successful
0812 02:36:53.963083 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":575,"thread_start_us":293,"threads_started":1}
I20260812 02:36:53.965656 15802 heartbeater.cc:347] Connected to a master server at 127.7.132.190:33723
I20260812 02:36:53.965986 15802 heartbeater.cc:464] Registering TS with master...
I20260812 02:36:53.966765 15802 heartbeater.cc:511] Master 127.7.132.190:33723 requested a full tablet report, sending...
I20260812 02:36:53.968369 15223 ts_manager.cc:194] Registered new tserver with Master: 6c404a3267bc4a11a6850d91dec7b21d (127.7.132.131:35367)
I20260812 02:36:53.969174 15223 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.132.131@KRBTEST.COM'} at 127.7.132.131:60725
I20260812 02:36:53.972508 7698 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.7.132.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.132.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.132.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/krb5kdc/oryx_127.7.132.189.keytab.
Entry for principal oryx/127.7.132.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/krb5kdc/oryx_127.7.132.189.keytab.
I20260812 02:36:54.030614 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.189:38815
--webserver_interface=127.7.132.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/krb5kdc/oryx_127.7.132.189.keytab
--principal=oryx/127.7.132.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:43909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.132.190:33723,127.7.132.189:38815
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:36:54.377672 15808 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:36:54.377957 15808 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:36:54.378160 15808 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:36:54.385461 15808 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:36:54.385556 15808 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:36:54.385681 15808 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:36:54.385779 15808 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:36:54.385854 15808 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:36:54.395651 15808 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:43909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.7.132.190:33723,127.7.132.189:38815
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/krb5kdc/oryx_127.7.132.189.keytab
--principal=oryx/127.7.132.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.189:38815
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.7.132.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
W20260812 02:36:54.396040 15808 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.7.132.190:33723,127.7.132.189:38815'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260812 02:36:54.397113 15808 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:36:54.398619 15808 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:36:54.410441 15815 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:36:54.410653 15816 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:36:54.410887 15818 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:36:54.411474 15808 server_base.cc:1034] running on GCE node
I20260812 02:36:54.412250 15808 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:36:54.413686 15808 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:36:54.415095 15808 hybrid_clock.cc:648] HybridClock initialized: now 1786502214414860 us; error 37 us; skew 500 ppm
I20260812 02:36:54.415468 15808 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:36:54 dist-test-slave-mn4r krb5kdc[15163](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502214, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.132.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:36:54.418841 15808 init.cc:377] Logged in from keytab as oryx/127.7.132.189@KRBTEST.COM (short username oryx)
I20260812 02:36:54.420302 15808 webserver.cc:460] Webserver started at http://127.7.132.189:43727/ using document root <none> and password file <none>
I20260812 02:36:54.421022 15808 fs_manager.cc:359] Metadata directory not provided
I20260812 02:36:54.421159 15808 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:36:54.421484 15808 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:36:54.424554 15808 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/master-1/data/instance:
uuid: "351974fae6dd4c168a844a01b53d9927"
format_stamp: "Formatted at 2026-08-12 02:36:54 on dist-test-slave-mn4r"
server_key: "5cee1bdc8c588bc83b89436a6bb21510"
server_key_iv: "422cea854e13da9dffc67afac03ed12a"
server_key_version: "encryptionkey@0"
I20260812 02:36:54.425379 15808 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/master-1/wal/instance:
uuid: "351974fae6dd4c168a844a01b53d9927"
format_stamp: "Formatted at 2026-08-12 02:36:54 on dist-test-slave-mn4r"
server_key: "5cee1bdc8c588bc83b89436a6bb21510"
server_key_iv: "422cea854e13da9dffc67afac03ed12a"
server_key_version: "encryptionkey@0"
I20260812 02:36:54.430835 15808 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260812 02:36:54.434628 15825 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:36:54.436044 15808 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260812 02:36:54.436251 15808 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/master-1/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/master-1/wal
uuid: "351974fae6dd4c168a844a01b53d9927"
format_stamp: "Formatted at 2026-08-12 02:36:54 on dist-test-slave-mn4r"
server_key: "5cee1bdc8c588bc83b89436a6bb21510"
server_key_iv: "422cea854e13da9dffc67afac03ed12a"
server_key_version: "encryptionkey@0"
I20260812 02:36:54.436478 15808 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:36:54.446343 15629 heartbeater.cc:503] Master 127.7.132.190:33723 was elected leader, sending a full tablet report...
I20260812 02:36:54.460536 15808 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:36:54.465047 15808 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:36:54.465341 15808 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:36:54.485131 15808 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.189:38815
I20260812 02:36:54.485170 15893 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.189:38815 every 8 connection(s)
I20260812 02:36:54.486720 15808 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/master-1/data/info.pb
I20260812 02:36:54.490665 15894 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:36:54.496546 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 15808
I20260812 02:36:54.497462 15894 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.132.190" port: 33723 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 02:36:54 dist-test-slave-mn4r krb5kdc[15163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502214, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.132.189@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM
Aug 12 02:36:54 dist-test-slave-mn4r krb5kdc[15163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502211, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.132.189@KRBTEST.COM
I20260812 02:36:54.518630 15900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:54.500585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42164 (local address 127.7.132.190:33723)
0812 02:36:54.502073 (+ 1488us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:54.502099 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:54.502124 (+ 25us) server_negotiation.cc:407] Connection header received
0812 02:36:54.502688 (+ 564us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:54.502696 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:54.502782 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:54.502906 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:54.504518 (+ 1612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:54.505294 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:54.506907 (+ 1613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:54.507322 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:54.507467 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:54.511441 (+ 3974us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:54.511446 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:54.511450 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:54.511485 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:54.514934 (+ 3449us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:54.515857 (+ 923us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:54.515864 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:54.515868 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:54.516005 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:54.516494 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:54.516497 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:54.516499 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:54.517010 (+ 511us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:54.517119 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:54.517956 (+ 837us) server_negotiation.cc:299] Negotiation successful
0812 02:36:54.518240 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1358,"thread_start_us":1244,"threads_started":1}
I20260812 02:36:54.519701 15899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:54.500531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:33723 (local address 127.0.0.1:42164)
0812 02:36:54.501535 (+ 1004us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:54.501622 (+ 87us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:54.502413 (+ 791us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:54.503404 (+ 991us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:54.503408 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:54.503979 (+ 571us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:36:54.504340 (+ 361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:54.504357 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:54.505448 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:54.505458 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:54.506759 (+ 1301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:54.506771 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:54.507577 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:54.507585 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:54.507826 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:54.508705 (+ 879us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:36:54.508732 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:36:54.511246 (+ 2514us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:36:54.515068 (+ 3822us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:54.515077 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:54.515087 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:54.515715 (+ 628us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:54.516121 (+ 406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:54.516126 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:54.516129 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:54.516372 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:54.517146 (+ 774us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:36:54.517157 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:36:54.517517 (+ 360us) client_negotiation.cc:769] Sending connection context
0812 02:36:54.517826 (+ 309us) client_negotiation.cc:240] Negotiation successful
0812 02:36:54.518388 (+ 562us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":596,"mutex_wait_us":61,"spinlock_wait_cycles":15104,"thread_start_us":355,"threads_started":1}
I20260812 02:36:54.519742 15898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:54.498963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39690 (local address 127.7.132.189:38815)
0812 02:36:54.499772 (+ 809us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:54.499783 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:54.499822 (+ 39us) server_negotiation.cc:407] Connection header received
0812 02:36:54.502398 (+ 2576us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:54.502421 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:54.502733 (+ 312us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:54.502945 (+ 212us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:54.504379 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:54.505567 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:54.506913 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:54.507444 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:54.507696 (+ 252us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:54.512137 (+ 4441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:54.512147 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:54.512159 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:54.512196 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:54.515757 (+ 3561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:54.516670 (+ 913us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:54.516680 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:54.516683 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:54.516823 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:54.517239 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:54.517242 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:54.517244 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:54.517588 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:54.517881 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:54.518294 (+ 413us) server_negotiation.cc:299] Negotiation successful
0812 02:36:54.518531 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":585,"thread_start_us":374,"threads_started":1}
I20260812 02:36:54.521749 15894 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.132.189" port: 38815 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 02:36:54 dist-test-slave-mn4r krb5kdc[15163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502214, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.132.189@KRBTEST.COM for oryx/127.7.132.189@KRBTEST.COM
I20260812 02:36:54.533531 15899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:54.522764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.189:38815 (local address 127.0.0.1:39702)
0812 02:36:54.522946 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:54.522961 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:54.523038 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:54.523274 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:54.523277 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:54.523583 (+ 306us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:36:54.523714 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:54.523721 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:54.524621 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:54.524624 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:54.525470 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:54.525480 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:54.526172 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:54.526176 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:54.526323 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:54.526819 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:36:54.526835 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:36:54.528729 (+ 1894us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:36:54.531386 (+ 2657us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:54.531390 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:54.531392 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:54.531745 (+ 353us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:54.532184 (+ 439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:54.532188 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:54.532190 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:54.532322 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:54.532839 (+ 517us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:36:54.532842 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:36:54.532959 (+ 117us) client_negotiation.cc:769] Sending connection context
0812 02:36:54.533067 (+ 108us) client_negotiation.cc:240] Negotiation successful
0812 02:36:54.533295 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63}
I20260812 02:36:54.533756 15898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:54.522829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39702 (local address 127.7.132.189:38815)
0812 02:36:54.522995 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:54.523000 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:54.523025 (+ 25us) server_negotiation.cc:407] Connection header received
0812 02:36:54.523110 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:54.523114 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:54.523164 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:54.523236 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:54.523831 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:54.524493 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:54.525588 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:54.525992 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:54.526146 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:54.528897 (+ 2751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:54.528902 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:54.528905 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:54.528932 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:54.531256 (+ 2324us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:54.531918 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:54.531926 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:54.531931 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:54.532068 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:54.532418 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:54.532421 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:54.532423 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:54.532739 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:54.532847 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:54.533154 (+ 307us) server_negotiation.cc:299] Negotiation successful
0812 02:36:54.533429 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":97,"server-negotiator.queue_time_us":52}
I20260812 02:36:54.538713 15894 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 351974fae6dd4c168a844a01b53d9927: Bootstrap starting.
I20260812 02:36:54.542054 15894 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 351974fae6dd4c168a844a01b53d9927: Neither blocks nor log segments found. Creating new log.
I20260812 02:36:54.543118 15894 log.cc:824] T 00000000000000000000000000000000 P 351974fae6dd4c168a844a01b53d9927: Log is configured to *not* fsync() on all Append() calls
I20260812 02:36:54.545586 15894 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 351974fae6dd4c168a844a01b53d9927: No bootstrap required, opened a new log
I20260812 02:36:54.555068 15894 raft_consensus.cc:348] T 00000000000000000000000000000000 P 351974fae6dd4c168a844a01b53d9927 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82e54874d347460c85c36da26b3bb8c8" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 33723 } } peers { permanent_uuid: "351974fae6dd4c168a844a01b53d9927" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 38815 } }
I20260812 02:36:54.555524 15894 raft_consensus.cc:374] T 00000000000000000000000000000000 P 351974fae6dd4c168a844a01b53d9927 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:36:54.555624 15894 raft_consensus.cc:729] T 00000000000000000000000000000000 P 351974fae6dd4c168a844a01b53d9927 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 351974fae6dd4c168a844a01b53d9927, State: Initialized, Role: FOLLOWER
I20260812 02:36:54.556355 15894 consensus_queue.cc:260] T 00000000000000000000000000000000 P 351974fae6dd4c168a844a01b53d9927 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82e54874d347460c85c36da26b3bb8c8" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 33723 } } peers { permanent_uuid: "351974fae6dd4c168a844a01b53d9927" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 38815 } }
I20260812 02:36:54.557355 15902 sys_catalog.cc:455] T 00000000000000000000000000000000 P 351974fae6dd4c168a844a01b53d9927 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82e54874d347460c85c36da26b3bb8c8" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 33723 } } peers { permanent_uuid: "351974fae6dd4c168a844a01b53d9927" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 38815 } } }
I20260812 02:36:54.557778 15902 sys_catalog.cc:458] T 00000000000000000000000000000000 P 351974fae6dd4c168a844a01b53d9927 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 02:36:54.558701 15894 sys_catalog.cc:564] T 00000000000000000000000000000000 P 351974fae6dd4c168a844a01b53d9927 [sys.catalog]: configured and running, proceeding with master startup.
W20260812 02:36:54.567780 15914 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 351974fae6dd4c168a844a01b53d9927: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 02:36:54.567934 15914 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 02:36:54.570938 15808 master_runner.cc:428] Detected that this master 351974fae6dd4c168a844a01b53d9927 is joining an existing cluster
I20260812 02:36:54.571038 15808 master_runner.cc:432] Initiating AddMaster RPC to add 127.7.132.189:38815
I20260812 02:36:54.592402 15900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:54.575358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42174 (local address 127.7.132.190:33723)
0812 02:36:54.575567 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:54.575574 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:54.576444 (+ 870us) server_negotiation.cc:407] Connection header received
0812 02:36:54.576710 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:54.576719 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:54.576824 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:54.576939 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:54.578309 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:54.579423 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:54.581543 (+ 2120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:54.582346 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:54.582452 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:54.585092 (+ 2640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:54.585099 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:54.585104 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:54.585156 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:54.588886 (+ 3730us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:54.589898 (+ 1012us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:54.589907 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:54.589912 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:54.590120 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:54.590593 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:54.590600 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:54.590604 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:54.590990 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:54.591148 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:54.591783 (+ 635us) server_negotiation.cc:299] Negotiation successful
0812 02:36:54.592033 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 02:36:54.593582 15920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:54.575630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:33723 (local address 127.0.0.1:42174)
0812 02:36:54.576338 (+ 708us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:54.576364 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:54.576551 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:54.576992 (+ 441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:54.576996 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:54.577696 (+ 700us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:36:54.578075 (+ 379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:54.578128 (+ 53us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:54.579616 (+ 1488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:54.579621 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:54.581373 (+ 1752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:54.581389 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:54.582747 (+ 1358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:54.582753 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:54.582920 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:54.583700 (+ 780us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:36:54.583725 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:36:54.584874 (+ 1149us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:36:54.589059 (+ 4185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:54.589068 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:54.589075 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:54.589740 (+ 665us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:54.590269 (+ 529us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:54.590274 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:54.590277 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:54.590458 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:54.591235 (+ 777us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:36:54.591244 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:36:54.591474 (+ 230us) client_negotiation.cc:769] Sending connection context
0812 02:36:54.592665 (+ 1191us) client_negotiation.cc:240] Negotiation successful
0812 02:36:54.593050 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":555,"thread_start_us":394,"threads_started":1}
I20260812 02:36:54.608075 15921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:54.578573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.189:38815 (local address 127.0.0.1:39710)
0812 02:36:54.590509 (+ 11936us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:54.590550 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:54.590700 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:54.590992 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:54.590996 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:54.591552 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:36:54.591793 (+ 241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:54.591804 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:54.593017 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:54.593021 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:54.594592 (+ 1571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:54.594613 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:54.595907 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:54.595914 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:54.596048 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:54.598263 (+ 2215us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:36:54.598290 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:36:54.599455 (+ 1165us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:36:54.604555 (+ 5100us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:54.604563 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:54.604569 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:54.605248 (+ 679us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:54.605740 (+ 492us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:54.605747 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:54.605753 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:54.605945 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:54.606780 (+ 835us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:36:54.606790 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:36:54.606958 (+ 168us) client_negotiation.cc:769] Sending connection context
0812 02:36:54.607113 (+ 155us) client_negotiation.cc:240] Negotiation successful
0812 02:36:54.607496 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11814,"thread_start_us":11639,"threads_started":1}
I20260812 02:36:54.608487 15902 raft_consensus.cc:482] T 00000000000000000000000000000000 P 351974fae6dd4c168a844a01b53d9927 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 02:36:54.608805 15902 raft_consensus.cc:504] T 00000000000000000000000000000000 P 351974fae6dd4c168a844a01b53d9927 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82e54874d347460c85c36da26b3bb8c8" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 33723 } } peers { permanent_uuid: "351974fae6dd4c168a844a01b53d9927" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 38815 } }
I20260812 02:36:54.609184 15898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:54.578956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39710 (local address 127.7.132.189:38815)
0812 02:36:54.579143 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:54.579150 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:54.590655 (+ 11505us) server_negotiation.cc:407] Connection header received
0812 02:36:54.590807 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:54.590811 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:54.590864 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:54.590967 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:54.591942 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:54.592797 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:54.594747 (+ 1950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:54.595609 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:54.595729 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:54.599637 (+ 3908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:54.599646 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:54.599653 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:54.599702 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:54.604339 (+ 4637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:54.605402 (+ 1063us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:54.605411 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:54.605417 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:54.605594 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:54.606099 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:54.606107 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:54.606112 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:54.606590 (+ 478us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:54.606728 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:54.608407 (+ 1679us) server_negotiation.cc:299] Negotiation successful
0812 02:36:54.608660 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 02:36:54.610365 15902 leader_election.cc:288] T 00000000000000000000000000000000 P 351974fae6dd4c168a844a01b53d9927 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 82e54874d347460c85c36da26b3bb8c8 (127.7.132.190:33723)
I20260812 02:36:54.611366 15235 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "351974fae6dd4c168a844a01b53d9927" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "82e54874d347460c85c36da26b3bb8c8" is_pre_election: true
I20260812 02:36:54.612538 15826 leader_election.cc:302] T 00000000000000000000000000000000 P 351974fae6dd4c168a844a01b53d9927 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: 351974fae6dd4c168a844a01b53d9927; no voters: 82e54874d347460c85c36da26b3bb8c8
I20260812 02:36:54.613307 15902 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 351974fae6dd4c168a844a01b53d9927 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
Aug 12 02:36:54 dist-test-slave-mn4r krb5kdc[15163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502212, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.132.190@KRBTEST.COM for oryx/127.7.132.189@KRBTEST.COM
WARNING: no policy specified for oryx/127.7.132.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.132.188@KRBTEST.COM" created.
I20260812 02:36:54.628981 15923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:54.600577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.189:38815 (local address 127.0.0.1:39720)
0812 02:36:54.608887 (+ 8310us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:54.608932 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:54.609118 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:54.611623 (+ 2505us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:54.611634 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:54.612586 (+ 952us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:36:54.612885 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:54.612906 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:54.614400 (+ 1494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:54.614406 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:54.615987 (+ 1581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:54.616004 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:54.617196 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:54.617204 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:54.617412 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:54.618225 (+ 813us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:36:54.618251 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:36:54.621267 (+ 3016us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:36:54.625689 (+ 4422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:54.625700 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:54.625706 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:54.626307 (+ 601us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:54.626768 (+ 461us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:54.626776 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:54.626781 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:54.626982 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:54.627686 (+ 704us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:36:54.627697 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:36:54.627924 (+ 227us) client_negotiation.cc:769] Sending connection context
0812 02:36:54.628170 (+ 246us) client_negotiation.cc:240] Negotiation successful
0812 02:36:54.628528 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8044,"thread_start_us":9922,"threads_started":1}
I20260812 02:36:54.629070 15898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:54.600639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39720 (local address 127.7.132.189:38815)
0812 02:36:54.611169 (+ 10530us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:54.611175 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:54.611191 (+ 16us) server_negotiation.cc:407] Connection header received
0812 02:36:54.611300 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:54.611305 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:54.611429 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:54.611539 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:54.613086 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:54.614223 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:54.616155 (+ 1932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:54.616998 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:54.617153 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:54.621486 (+ 4333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:54.621492 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:54.621497 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:54.621535 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:54.625473 (+ 3938us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:54.626482 (+ 1009us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:54.626487 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:54.626490 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:54.626616 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:54.627128 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:54.627136 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:54.627140 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:54.627510 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:54.627661 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:54.628298 (+ 637us) server_negotiation.cc:299] Negotiation successful
0812 02:36:54.628568 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10423,"thread_start_us":13650,"threads_started":1}
I20260812 02:36:54.634990 15223 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.7.132.189:38815: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "351974fae6dd4c168a844a01b53d9927"
member_type: NON_VOTER
last_known_addr {
host: "127.7.132.189"
port: 38815
}
attrs {
promote: true
}
}
dest_uuid: "82e54874d347460c85c36da26b3bb8c8"
cas_config_opid_index: -1
I20260812 02:36:54.636411 15223 consensus_queue.cc:237] T 00000000000000000000000000000000 P 82e54874d347460c85c36da26b3bb8c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "82e54874d347460c85c36da26b3bb8c8" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 33723 } } peers { permanent_uuid: "351974fae6dd4c168a844a01b53d9927" member_type: NON_VOTER last_known_addr { host: "127.7.132.189" port: 38815 } attrs { promote: true } }
I20260812 02:36:54.647666 15856 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 351974fae6dd4c168a844a01b53d9927 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:36:54.650688 15928 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 82e54874d347460c85c36da26b3bb8c8 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 351974fae6dd4c168a844a01b53d9927 (127.7.132.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "82e54874d347460c85c36da26b3bb8c8" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 33723 } } peers { permanent_uuid: "351974fae6dd4c168a844a01b53d9927" member_type: NON_VOTER last_known_addr { host: "127.7.132.189" port: 38815 } attrs { promote: true } } }
I20260812 02:36:54.651659 15856 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 351974fae6dd4c168a844a01b53d9927 [term 1 FOLLOWER]: Refusing update from remote peer 82e54874d347460c85c36da26b3bb8c8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260812 02:36:54.652715 15929 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 82e54874d347460c85c36da26b3bb8c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "351974fae6dd4c168a844a01b53d9927" member_type: NON_VOTER last_known_addr { host: "127.7.132.189" port: 38815 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:36:54.657704 15928 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.7.132.189:38815
I20260812 02:36:54.660602 15808 master.cc:537] Master@127.7.132.189:38815 shutting down...
I20260812 02:36:54.661154 15930 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82e54874d347460c85c36da26b3bb8c8 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "82e54874d347460c85c36da26b3bb8c8" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "82e54874d347460c85c36da26b3bb8c8" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 33723 } } peers { permanent_uuid: "351974fae6dd4c168a844a01b53d9927" member_type: NON_VOTER last_known_addr { host: "127.7.132.189" port: 38815 } attrs { promote: true } } }
I20260812 02:36:54.661599 15930 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82e54874d347460c85c36da26b3bb8c8 [sys.catalog]: This master's current role is: LEADER
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.132.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/krb5kdc/oryx_127.7.132.188.keytab.
Entry for principal oryx/127.7.132.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/krb5kdc/oryx_127.7.132.188.keytab.
I20260812 02:36:54.670730 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.188:39485
--webserver_interface=127.7.132.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/krb5kdc/oryx_127.7.132.188.keytab
--principal=oryx/127.7.132.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:43909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.132.190:33723,127.7.132.189:38815,127.7.132.188:39485
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 02:36:54.683861 15856 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 351974fae6dd4c168a844a01b53d9927 [term 1 LEARNER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 351974fae6dd4c168a844a01b53d9927 (127.7.132.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "82e54874d347460c85c36da26b3bb8c8" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 33723 } } peers { permanent_uuid: "351974fae6dd4c168a844a01b53d9927" member_type: NON_VOTER last_known_addr { host: "127.7.132.189" port: 38815 } attrs { promote: true } } }
I20260812 02:36:54.702419 15942 sys_catalog.cc:455] T 00000000000000000000000000000000 P 351974fae6dd4c168a844a01b53d9927 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "82e54874d347460c85c36da26b3bb8c8" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "82e54874d347460c85c36da26b3bb8c8" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 33723 } } peers { permanent_uuid: "351974fae6dd4c168a844a01b53d9927" member_type: NON_VOTER last_known_addr { host: "127.7.132.189" port: 38815 } attrs { promote: true } } }
I20260812 02:36:54.702981 15942 sys_catalog.cc:458] T 00000000000000000000000000000000 P 351974fae6dd4c168a844a01b53d9927 [sys.catalog]: This master's current role is: LEARNER
I20260812 02:36:54.703231 15937 sys_catalog.cc:455] T 00000000000000000000000000000000 P 351974fae6dd4c168a844a01b53d9927 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "82e54874d347460c85c36da26b3bb8c8" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "82e54874d347460c85c36da26b3bb8c8" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 33723 } } peers { permanent_uuid: "351974fae6dd4c168a844a01b53d9927" member_type: NON_VOTER last_known_addr { host: "127.7.132.189" port: 38815 } attrs { promote: true } } }
I20260812 02:36:54.703625 15937 sys_catalog.cc:458] T 00000000000000000000000000000000 P 351974fae6dd4c168a844a01b53d9927 [sys.catalog]: This master's current role is: LEARNER
I20260812 02:36:54.702982 15902 sys_catalog.cc:455] T 00000000000000000000000000000000 P 351974fae6dd4c168a844a01b53d9927 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 82e54874d347460c85c36da26b3bb8c8. Latest consensus state: current_term: 1 leader_uuid: "82e54874d347460c85c36da26b3bb8c8" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "82e54874d347460c85c36da26b3bb8c8" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 33723 } } peers { permanent_uuid: "351974fae6dd4c168a844a01b53d9927" member_type: NON_VOTER last_known_addr { host: "127.7.132.189" port: 38815 } attrs { promote: true } } }
I20260812 02:36:54.703974 15902 sys_catalog.cc:458] T 00000000000000000000000000000000 P 351974fae6dd4c168a844a01b53d9927 [sys.catalog]: This master's current role is: LEARNER
I20260812 02:36:54.706511 15808 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 351974fae6dd4c168a844a01b53d9927 [term 1 LEARNER]: Raft consensus shutting down.
I20260812 02:36:54.707059 15808 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 351974fae6dd4c168a844a01b53d9927 [term 1 LEARNER]: Raft consensus is shut down!
I20260812 02:36:54.707216 15808 tablet_replica.cc:330] T 00000000000000000000000000000000 P 351974fae6dd4c168a844a01b53d9927: stopping tablet replica
I20260812 02:36:54.715812 15808 master.cc:552] Master@127.7.132.189:38815 shutdown complete.
I20260812 02:36:54.715973 15808 master_runner.cc:305] Clearing existing system tablet
I20260812 02:36:54.716413 15808 fs_manager.cc:359] Metadata directory not provided
I20260812 02:36:54.716526 15808 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:36:54.718855 15808 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260812 02:36:54.720547 15944 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:36:54.720978 15808 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:36:54.721230 15808 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 02:36:54.721315 15808 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/master-1/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/master-1/wal
uuid: "351974fae6dd4c168a844a01b53d9927"
format_stamp: "Formatted at 2026-08-12 02:36:54 on dist-test-slave-mn4r"
server_key: "5cee1bdc8c588bc83b89436a6bb21510"
server_key_iv: "422cea854e13da9dffc67afac03ed12a"
server_key_version: "encryptionkey@0"
I20260812 02:36:54.724589 15808 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 351974fae6dd4c168a844a01b53d9927: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:36:54.729177 15808 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 351974fae6dd4c168a844a01b53d9927: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260812 02:36:54.729364 15808 log.cc:1196] T 00000000000000000000000000000000 P 351974fae6dd4c168a844a01b53d9927: Deleting WAL directory at /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260812 02:36:54.729794 15808 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 351974fae6dd4c168a844a01b53d9927: Deleting consensus metadata
I20260812 02:36:54.730126 15808 master_runner.cc:315] Copying system tablet from 127.7.132.190:33723
I20260812 02:36:54.733853 15808 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 351974fae6dd4c168a844a01b53d9927: tablet copy: Beginning tablet copy session from remote peer at address 127.7.132.190:33723
I20260812 02:36:54.747337 15900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:54.734849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42184 (local address 127.7.132.190:33723)
0812 02:36:54.735087 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:54.735096 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:54.735637 (+ 541us) server_negotiation.cc:407] Connection header received
0812 02:36:54.735848 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:54.735855 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:54.735951 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:54.736063 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:54.736883 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:54.737632 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:54.739076 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:54.739586 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:54.739662 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:54.741742 (+ 2080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:54.741748 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:54.741754 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:54.741794 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:54.744555 (+ 2761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:54.745194 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:54.745200 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:54.745204 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:54.745376 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:54.745775 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:54.745783 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:54.745786 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:54.746070 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:54.746260 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:54.746730 (+ 470us) server_negotiation.cc:299] Negotiation successful
0812 02:36:54.746941 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260812 02:36:54.747407 15953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:54.734813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:33723 (local address 127.0.0.1:42184)
0812 02:36:54.735512 (+ 699us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:54.735534 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:54.735705 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:54.736091 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:54.736096 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:54.736520 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:36:54.736718 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:54.736727 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:54.737797 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:54.737802 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:54.738860 (+ 1058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:54.738878 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:54.739895 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:54.739900 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:54.740048 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:54.740709 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:36:54.740727 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:36:54.741542 (+ 815us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:36:54.744694 (+ 3152us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:54.744700 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:54.744704 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:54.745062 (+ 358us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:54.745503 (+ 441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:54.745508 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:54.745511 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:54.745663 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:54.746220 (+ 557us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:36:54.746225 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:36:54.746393 (+ 168us) client_negotiation.cc:769] Sending connection context
0812 02:36:54.746619 (+ 226us) client_negotiation.cc:240] Negotiation successful
0812 02:36:54.746966 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":556,"thread_start_us":417,"threads_started":1}
I20260812 02:36:54.748543 15246 tablet_copy_service.cc:140] P 82e54874d347460c85c36da26b3bb8c8: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 351974fae6dd4c168a844a01b53d9927 ({username='oryx', principal='oryx/127.7.132.189@KRBTEST.COM'} at 127.0.0.1:42184)
I20260812 02:36:54.748925 15246 tablet_copy_service.cc:161] P 82e54874d347460c85c36da26b3bb8c8: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 351974fae6dd4c168a844a01b53d9927 at {username='oryx', principal='oryx/127.7.132.189@KRBTEST.COM'} at 127.0.0.1:42184: session id = 351974fae6dd4c168a844a01b53d9927-00000000000000000000000000000000
I20260812 02:36:54.752372 15246 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 82e54874d347460c85c36da26b3bb8c8: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 02:36:54.754565 15808 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260812 02:36:54.759829 15808 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 351974fae6dd4c168a844a01b53d9927: tablet copy: Starting download of 0 data blocks...
I20260812 02:36:54.760251 15808 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 351974fae6dd4c168a844a01b53d9927: tablet copy: Starting download of 1 WAL segments...
I20260812 02:36:54.763376 15808 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 351974fae6dd4c168a844a01b53d9927: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 02:36:54.766723 15246 tablet_copy_service.cc:342] P 82e54874d347460c85c36da26b3bb8c8: Request end of tablet copy session 351974fae6dd4c168a844a01b53d9927-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.7.132.189@KRBTEST.COM'} at 127.0.0.1:42184
I20260812 02:36:54.766957 15246 tablet_copy_service.cc:433] P 82e54874d347460c85c36da26b3bb8c8: ending tablet copy session 351974fae6dd4c168a844a01b53d9927-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 351974fae6dd4c168a844a01b53d9927
I20260812 02:36:54.777894 15808 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:36:54.780790 15954 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:36:54.790444 15955 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:36:54.790791 15957 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:36:54.795434 15808 server_base.cc:1034] running on GCE node
I20260812 02:36:54.795774 15808 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:36:54.796507 15808 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:36:54.797751 15808 hybrid_clock.cc:648] HybridClock initialized: now 1786502214797692 us; error 30 us; skew 500 ppm
I20260812 02:36:54.798055 15808 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:36:54 dist-test-slave-mn4r krb5kdc[15163](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502214, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.132.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:36:54.800060 15808 init.cc:377] Logged in from keytab as oryx/127.7.132.189@KRBTEST.COM (short username oryx)
I20260812 02:36:54.801378 15808 webserver.cc:460] Webserver started at http://127.7.132.189:33295/ using document root <none> and password file <none>
I20260812 02:36:54.801710 15808 fs_manager.cc:359] Metadata directory not provided
I20260812 02:36:54.801807 15808 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:36:54.804432 15808 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.001s
I20260812 02:36:54.806453 15964 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:36:54.806931 15808 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260812 02:36:54.807106 15808 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/master-1/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/master-1/wal
uuid: "351974fae6dd4c168a844a01b53d9927"
format_stamp: "Formatted at 2026-08-12 02:36:54 on dist-test-slave-mn4r"
server_key: "5cee1bdc8c588bc83b89436a6bb21510"
server_key_iv: "422cea854e13da9dffc67afac03ed12a"
server_key_version: "encryptionkey@0"
I20260812 02:36:54.807271 15808 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:36:54.817713 15808 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:36:54.821686 15808 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:36:54.856549 15808 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.189:38815
I20260812 02:36:54.856571 16023 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.189:38815 every 8 connection(s)
I20260812 02:36:54.857687 15808 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/master-1/data/info.pb
I20260812 02:36:54.865744 16024 sys_catalog.cc:263] Verifying existing consensus state
I20260812 02:36:54.871701 16024 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 351974fae6dd4c168a844a01b53d9927: Bootstrap starting.
I20260812 02:36:54.893626 16024 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 351974fae6dd4c168a844a01b53d9927: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 02:36:54.894151 16024 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 351974fae6dd4c168a844a01b53d9927: Bootstrap complete.
I20260812 02:36:54.895080 16024 raft_consensus.cc:348] T 00000000000000000000000000000000 P 351974fae6dd4c168a844a01b53d9927 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "82e54874d347460c85c36da26b3bb8c8" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 33723 } } peers { permanent_uuid: "351974fae6dd4c168a844a01b53d9927" member_type: NON_VOTER last_known_addr { host: "127.7.132.189" port: 38815 } attrs { promote: true } }
I20260812 02:36:54.895308 16024 raft_consensus.cc:729] T 00000000000000000000000000000000 P 351974fae6dd4c168a844a01b53d9927 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 351974fae6dd4c168a844a01b53d9927, State: Initialized, Role: LEARNER
I20260812 02:36:54.895591 16024 consensus_queue.cc:260] T 00000000000000000000000000000000 P 351974fae6dd4c168a844a01b53d9927 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "82e54874d347460c85c36da26b3bb8c8" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 33723 } } peers { permanent_uuid: "351974fae6dd4c168a844a01b53d9927" member_type: NON_VOTER last_known_addr { host: "127.7.132.189" port: 38815 } attrs { promote: true } }
I20260812 02:36:54.896448 16027 sys_catalog.cc:455] T 00000000000000000000000000000000 P 351974fae6dd4c168a844a01b53d9927 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "82e54874d347460c85c36da26b3bb8c8" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 33723 } } peers { permanent_uuid: "351974fae6dd4c168a844a01b53d9927" member_type: NON_VOTER last_known_addr { host: "127.7.132.189" port: 38815 } attrs { promote: true } } }
I20260812 02:36:54.896795 16024 sys_catalog.cc:564] T 00000000000000000000000000000000 P 351974fae6dd4c168a844a01b53d9927 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:36:54.896853 16027 sys_catalog.cc:458] T 00000000000000000000000000000000 P 351974fae6dd4c168a844a01b53d9927 [sys.catalog]: This master's current role is: LEARNER
I20260812 02:36:54.908923 16040 catalog_manager.cc:1259] Loaded cluster ID: eb5dad367dc04d1ba16a1cfb4c2dc8a7
I20260812 02:36:54.909058 16040 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 351974fae6dd4c168a844a01b53d9927: loading cluster ID for follower catalog manager: success
I20260812 02:36:54.911692 16040 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 351974fae6dd4c168a844a01b53d9927: acquiring CA information for follower catalog manager: success
I20260812 02:36:54.913604 16040 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 351974fae6dd4c168a844a01b53d9927: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 02:36:54.972205 15802 heartbeater.cc:503] Master 127.7.132.190:33723 was elected leader, sending a full tablet report...
I20260812 02:36:55.069330 15930 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82e54874d347460c85c36da26b3bb8c8 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "82e54874d347460c85c36da26b3bb8c8" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "82e54874d347460c85c36da26b3bb8c8" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 33723 } } peers { permanent_uuid: "351974fae6dd4c168a844a01b53d9927" member_type: NON_VOTER last_known_addr { host: "127.7.132.189" port: 38815 } attrs { promote: true } } }
I20260812 02:36:55.069649 15930 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82e54874d347460c85c36da26b3bb8c8 [sys.catalog]: This master's current role is: LEADER
I20260812 02:36:55.079182 15923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:55.069523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.189:38815 (local address 127.0.0.1:39724)
0812 02:36:55.069946 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:55.069967 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:55.070129 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:55.071674 (+ 1545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:55.071680 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:55.071717 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:36:55.071964 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:55.071977 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:55.074124 (+ 2147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:55.074133 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:55.076260 (+ 2127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:55.076277 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:55.078226 (+ 1949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:55.078232 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:55.078481 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:55.078553 (+ 72us) client_negotiation.cc:769] Sending connection context
0812 02:36:55.078676 (+ 123us) client_negotiation.cc:240] Negotiation successful
0812 02:36:55.078846 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285}
I20260812 02:36:55.079314 16041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:55.070437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39724 (local address 127.7.132.189:38815)
0812 02:36:55.071190 (+ 753us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:55.071196 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:55.071221 (+ 25us) server_negotiation.cc:407] Connection header received
0812 02:36:55.071305 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:55.071310 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:55.071422 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:55.071582 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:36:55.072147 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:55.073893 (+ 1746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:55.076517 (+ 2624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:55.077977 (+ 1460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:55.078150 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:55.078428 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:55.078722 (+ 294us) server_negotiation.cc:299] Negotiation successful
0812 02:36:55.078873 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":611,"thread_start_us":415,"threads_started":1}
I20260812 02:36:55.081029 16027 sys_catalog.cc:455] T 00000000000000000000000000000000 P 351974fae6dd4c168a844a01b53d9927 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 82e54874d347460c85c36da26b3bb8c8. Latest consensus state: current_term: 1 leader_uuid: "82e54874d347460c85c36da26b3bb8c8" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "82e54874d347460c85c36da26b3bb8c8" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 33723 } } peers { permanent_uuid: "351974fae6dd4c168a844a01b53d9927" member_type: NON_VOTER last_known_addr { host: "127.7.132.189" port: 38815 } attrs { promote: true } } }
I20260812 02:36:55.081348 16027 sys_catalog.cc:458] T 00000000000000000000000000000000 P 351974fae6dd4c168a844a01b53d9927 [sys.catalog]: This master's current role is: LEARNER
I20260812 02:36:55.082522 15929 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 82e54874d347460c85c36da26b3bb8c8: attempting to promote NON_VOTER 351974fae6dd4c168a844a01b53d9927 to VOTER
I20260812 02:36:55.083490 15929 consensus_queue.cc:237] T 00000000000000000000000000000000 P 82e54874d347460c85c36da26b3bb8c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "82e54874d347460c85c36da26b3bb8c8" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 33723 } } peers { permanent_uuid: "351974fae6dd4c168a844a01b53d9927" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 38815 } attrs { promote: false } }
I20260812 02:36:55.085484 15998 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 351974fae6dd4c168a844a01b53d9927 [term 1 LEARNER]: Refusing update from remote peer 82e54874d347460c85c36da26b3bb8c8: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260812 02:36:55.086302 15928 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 82e54874d347460c85c36da26b3bb8c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "351974fae6dd4c168a844a01b53d9927" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 38815 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260812 02:36:55.089874 15929 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 82e54874d347460c85c36da26b3bb8c8 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 351974fae6dd4c168a844a01b53d9927 (127.7.132.189) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "82e54874d347460c85c36da26b3bb8c8" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 33723 } } peers { permanent_uuid: "351974fae6dd4c168a844a01b53d9927" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 38815 } attrs { promote: false } } }
I20260812 02:36:55.090673 15998 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 351974fae6dd4c168a844a01b53d9927 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 351974fae6dd4c168a844a01b53d9927 (127.7.132.189) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "82e54874d347460c85c36da26b3bb8c8" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 33723 } } peers { permanent_uuid: "351974fae6dd4c168a844a01b53d9927" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 38815 } attrs { promote: false } } }
I20260812 02:36:55.093647 15929 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82e54874d347460c85c36da26b3bb8c8 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "82e54874d347460c85c36da26b3bb8c8" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "82e54874d347460c85c36da26b3bb8c8" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 33723 } } peers { permanent_uuid: "351974fae6dd4c168a844a01b53d9927" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 38815 } attrs { promote: false } } }
I20260812 02:36:55.093871 15929 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82e54874d347460c85c36da26b3bb8c8 [sys.catalog]: This master's current role is: LEADER
I20260812 02:36:55.093897 15930 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82e54874d347460c85c36da26b3bb8c8 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "82e54874d347460c85c36da26b3bb8c8" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "82e54874d347460c85c36da26b3bb8c8" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 33723 } } peers { permanent_uuid: "351974fae6dd4c168a844a01b53d9927" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 38815 } attrs { promote: false } } }
I20260812 02:36:55.094157 15930 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82e54874d347460c85c36da26b3bb8c8 [sys.catalog]: This master's current role is: LEADER
I20260812 02:36:55.096529 16027 sys_catalog.cc:455] T 00000000000000000000000000000000 P 351974fae6dd4c168a844a01b53d9927 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "82e54874d347460c85c36da26b3bb8c8" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "82e54874d347460c85c36da26b3bb8c8" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 33723 } } peers { permanent_uuid: "351974fae6dd4c168a844a01b53d9927" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 38815 } attrs { promote: false } } }
I20260812 02:36:55.096786 16027 sys_catalog.cc:458] T 00000000000000000000000000000000 P 351974fae6dd4c168a844a01b53d9927 [sys.catalog]: This master's current role is: FOLLOWER
W20260812 02:36:55.135126 15940 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:36:55.135476 15940 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:36:55.135635 15940 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:36:55.143218 15940 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:36:55.143364 15940 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:36:55.143472 15940 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:36:55.143577 15940 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:36:55.143653 15940 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:36:55.154145 15940 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:43909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.7.132.190:33723,127.7.132.189:38815,127.7.132.188:39485
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/krb5kdc/oryx_127.7.132.188.keytab
--principal=oryx/127.7.132.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.188:39485
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.7.132.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:36:55.155743 15940 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:36:55.157341 15940 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:36:55.169653 16048 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:36:55.169643 16049 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:36:55.170282 16054 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:36:55.171409 15940 server_base.cc:1034] running on GCE node
I20260812 02:36:55.172235 15940 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:36:55.173817 15940 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:36:55.175056 15940 hybrid_clock.cc:648] HybridClock initialized: now 1786502215174993 us; error 60 us; skew 500 ppm
I20260812 02:36:55.175452 15940 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:36:55 dist-test-slave-mn4r krb5kdc[15163](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502215, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.132.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:36:55.178864 15940 init.cc:377] Logged in from keytab as oryx/127.7.132.188@KRBTEST.COM (short username oryx)
I20260812 02:36:55.180544 15940 webserver.cc:460] Webserver started at http://127.7.132.188:40435/ using document root <none> and password file <none>
I20260812 02:36:55.181205 15940 fs_manager.cc:359] Metadata directory not provided
I20260812 02:36:55.181382 15940 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:36:55.181748 15940 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:36:55.185498 15940 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/master-2/data/instance:
uuid: "a6d878afb46e4df18827ac19aa9f97bd"
format_stamp: "Formatted at 2026-08-12 02:36:55 on dist-test-slave-mn4r"
server_key: "c4f21b60af8c39038bb7cb8311d650ce"
server_key_iv: "b199d2cc1a0a458e39ae94a9d9e4eb03"
server_key_version: "encryptionkey@0"
I20260812 02:36:55.186386 15940 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/master-2/wal/instance:
uuid: "a6d878afb46e4df18827ac19aa9f97bd"
format_stamp: "Formatted at 2026-08-12 02:36:55 on dist-test-slave-mn4r"
server_key: "c4f21b60af8c39038bb7cb8311d650ce"
server_key_iv: "b199d2cc1a0a458e39ae94a9d9e4eb03"
server_key_version: "encryptionkey@0"
I20260812 02:36:55.191931 15940 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260812 02:36:55.195653 16061 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:36:55.197230 15940 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260812 02:36:55.197419 15940 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/master-2/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/master-2/wal
uuid: "a6d878afb46e4df18827ac19aa9f97bd"
format_stamp: "Formatted at 2026-08-12 02:36:55 on dist-test-slave-mn4r"
server_key: "c4f21b60af8c39038bb7cb8311d650ce"
server_key_iv: "b199d2cc1a0a458e39ae94a9d9e4eb03"
server_key_version: "encryptionkey@0"
I20260812 02:36:55.197686 15940 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:36:55.214974 15940 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:36:55.219547 15940 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:36:55.219843 15940 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:36:55.239305 15940 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.188:39485
I20260812 02:36:55.239336 16122 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.188:39485 every 8 connection(s)
I20260812 02:36:55.240723 15940 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/master-2/data/info.pb
I20260812 02:36:55.241621 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 15940
I20260812 02:36:55.245517 16123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:36:55.254621 16123 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.132.190" port: 33723 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 02:36:55 dist-test-slave-mn4r krb5kdc[15163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502211, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.132.188@KRBTEST.COM
Aug 12 02:36:55 dist-test-slave-mn4r krb5kdc[15163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502215, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.132.188@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM
I20260812 02:36:55.268646 16126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:55.244041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51736 (local address 127.7.132.188:39485)
0812 02:36:55.244852 (+ 811us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:55.244867 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:55.244917 (+ 50us) server_negotiation.cc:407] Connection header received
0812 02:36:55.247294 (+ 2377us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:55.247319 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:55.247798 (+ 479us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:55.248237 (+ 439us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:55.249616 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:55.251769 (+ 2153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:55.253293 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:55.254398 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:55.254590 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:55.260207 (+ 5617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:55.260218 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:55.260231 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:55.260277 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:55.264063 (+ 3786us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:55.264896 (+ 833us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:55.264905 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:55.264913 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:55.265149 (+ 236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:55.265547 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:55.265553 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:55.265558 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:55.266126 (+ 568us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:55.266499 (+ 373us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:55.267035 (+ 536us) server_negotiation.cc:299] Negotiation successful
0812 02:36:55.267422 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":588,"thread_start_us":333,"threads_started":1}
I20260812 02:36:55.274365 16127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:55.257751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:33723 (local address 127.0.0.1:42196)
0812 02:36:55.258642 (+ 891us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:55.258686 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:55.258842 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:55.259579 (+ 737us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:55.259584 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:55.260348 (+ 764us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:36:55.260728 (+ 380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:55.260745 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:55.262076 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:55.262082 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:55.263937 (+ 1855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:55.263955 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:55.265000 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:55.265010 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:55.265218 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:55.265849 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:36:55.265866 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:36:55.268185 (+ 2319us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:36:55.272060 (+ 3875us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:55.272068 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:55.272072 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:55.272495 (+ 423us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:55.272850 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:55.272853 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:55.272855 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:55.272994 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:55.273486 (+ 492us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:36:55.273491 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:36:55.273631 (+ 140us) client_negotiation.cc:769] Sending connection context
0812 02:36:55.273772 (+ 141us) client_negotiation.cc:240] Negotiation successful
0812 02:36:55.274012 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":572,"thread_start_us":425,"threads_started":1}
I20260812 02:36:55.274482 16128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:55.257885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42196 (local address 127.7.132.190:33723)
0812 02:36:55.259188 (+ 1303us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:55.259195 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:55.259214 (+ 19us) server_negotiation.cc:407] Connection header received
0812 02:36:55.259279 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:55.259286 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:55.259370 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:55.259514 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:55.260964 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:55.261881 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:55.264092 (+ 2211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:55.264806 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:55.264928 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:55.268377 (+ 3449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:55.268383 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:55.268388 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:55.268429 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:55.271942 (+ 3513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:55.272609 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:55.272613 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:55.272615 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:55.272757 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:55.273102 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:55.273105 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:55.273107 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:55.273378 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:55.273491 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:55.273884 (+ 393us) server_negotiation.cc:299] Negotiation successful
0812 02:36:55.274053 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1179,"thread_start_us":1058,"threads_started":1}
I20260812 02:36:55.275767 16123 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.132.189" port: 38815 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 02:36:55 dist-test-slave-mn4r krb5kdc[15163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502215, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.132.188@KRBTEST.COM for oryx/127.7.132.189@KRBTEST.COM
I20260812 02:36:55.288528 16127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:55.276561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.189:38815 (local address 127.0.0.1:39740)
0812 02:36:55.276782 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:55.276802 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:55.276944 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:55.277267 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:55.277272 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:55.277637 (+ 365us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:36:55.277770 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:55.277777 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:55.278920 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:55.278925 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:55.279844 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:55.279856 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:55.280590 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:55.280596 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:55.280767 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:55.281285 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:36:55.281300 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:36:55.283371 (+ 2071us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:36:55.286145 (+ 2774us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:55.286151 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:55.286155 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:55.286527 (+ 372us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:55.286919 (+ 392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:55.286924 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:55.286928 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:55.287064 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:55.287582 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:36:55.287587 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:36:55.287736 (+ 149us) client_negotiation.cc:769] Sending connection context
0812 02:36:55.287877 (+ 141us) client_negotiation.cc:240] Negotiation successful
0812 02:36:55.288180 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260812 02:36:55.288551 16041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:55.276600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39740 (local address 127.7.132.189:38815)
0812 02:36:55.276785 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:55.276792 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:55.276881 (+ 89us) server_negotiation.cc:407] Connection header received
0812 02:36:55.277071 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:55.277075 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:55.277140 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:55.277246 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:55.277922 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:55.278772 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:55.279986 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:55.280412 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:55.280544 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:55.283557 (+ 3013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:55.283562 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:55.283565 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:55.283596 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:55.285980 (+ 2384us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:55.286659 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:55.286668 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:55.286671 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:55.286801 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:55.287180 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:55.287187 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:55.287190 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:55.287457 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:55.287638 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:55.287981 (+ 343us) server_negotiation.cc:299] Negotiation successful
0812 02:36:55.288217 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 02:36:55.289716 16123 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.132.188" port: 39485 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 02:36:55 dist-test-slave-mn4r krb5kdc[15163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502215, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.132.188@KRBTEST.COM for oryx/127.7.132.188@KRBTEST.COM
I20260812 02:36:55.301513 16127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:55.290392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:39485 (local address 127.0.0.1:51742)
0812 02:36:55.290598 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:55.290618 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:55.290747 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:55.291048 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:55.291052 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:55.291404 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:36:55.291533 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:55.291541 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:55.292456 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:55.292464 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:55.293276 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:55.293287 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:55.293958 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:55.293964 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:55.294192 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:55.294724 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:36:55.294740 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:36:55.296624 (+ 1884us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:36:55.299440 (+ 2816us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:55.299444 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:55.299446 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:55.299793 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:55.300112 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:55.300115 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:55.300117 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:55.300256 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:55.300741 (+ 485us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:36:55.300744 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:36:55.300864 (+ 120us) client_negotiation.cc:769] Sending connection context
0812 02:36:55.300993 (+ 129us) client_negotiation.cc:240] Negotiation successful
0812 02:36:55.301169 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260812 02:36:55.301622 16126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:55.290442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51742 (local address 127.7.132.188:39485)
0812 02:36:55.290598 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:55.290604 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:55.290689 (+ 85us) server_negotiation.cc:407] Connection header received
0812 02:36:55.290848 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:55.290853 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:55.290914 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:55.291035 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:55.291660 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:55.292335 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:55.293418 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:55.293807 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:55.293941 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:55.296832 (+ 2891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:55.296837 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:55.296840 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:55.296867 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:55.299316 (+ 2449us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:55.299891 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:55.299894 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:55.299896 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:55.300010 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:55.300345 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:55.300349 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:55.300351 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:55.300634 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:55.300754 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:55.301068 (+ 314us) server_negotiation.cc:299] Negotiation successful
0812 02:36:55.301294 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":40,"server-negotiator.queue_time_us":46}
I20260812 02:36:55.306550 16123 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6d878afb46e4df18827ac19aa9f97bd: Bootstrap starting.
I20260812 02:36:55.309798 16123 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a6d878afb46e4df18827ac19aa9f97bd: Neither blocks nor log segments found. Creating new log.
I20260812 02:36:55.311142 16123 log.cc:824] T 00000000000000000000000000000000 P a6d878afb46e4df18827ac19aa9f97bd: Log is configured to *not* fsync() on all Append() calls
I20260812 02:36:55.313879 16123 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6d878afb46e4df18827ac19aa9f97bd: No bootstrap required, opened a new log
I20260812 02:36:55.321836 16123 raft_consensus.cc:348] T 00000000000000000000000000000000 P a6d878afb46e4df18827ac19aa9f97bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82e54874d347460c85c36da26b3bb8c8" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 33723 } } peers { permanent_uuid: "351974fae6dd4c168a844a01b53d9927" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 38815 } } peers { permanent_uuid: "a6d878afb46e4df18827ac19aa9f97bd" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 39485 } }
I20260812 02:36:55.322290 16123 raft_consensus.cc:374] T 00000000000000000000000000000000 P a6d878afb46e4df18827ac19aa9f97bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:36:55.322445 16123 raft_consensus.cc:729] T 00000000000000000000000000000000 P a6d878afb46e4df18827ac19aa9f97bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6d878afb46e4df18827ac19aa9f97bd, State: Initialized, Role: FOLLOWER
I20260812 02:36:55.323189 16123 consensus_queue.cc:260] T 00000000000000000000000000000000 P a6d878afb46e4df18827ac19aa9f97bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82e54874d347460c85c36da26b3bb8c8" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 33723 } } peers { permanent_uuid: "351974fae6dd4c168a844a01b53d9927" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 38815 } } peers { permanent_uuid: "a6d878afb46e4df18827ac19aa9f97bd" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 39485 } }
I20260812 02:36:55.324169 16130 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6d878afb46e4df18827ac19aa9f97bd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82e54874d347460c85c36da26b3bb8c8" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 33723 } } peers { permanent_uuid: "351974fae6dd4c168a844a01b53d9927" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 38815 } } peers { permanent_uuid: "a6d878afb46e4df18827ac19aa9f97bd" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 39485 } } }
I20260812 02:36:55.324482 16130 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6d878afb46e4df18827ac19aa9f97bd [sys.catalog]: This master's current role is: FOLLOWER
I20260812 02:36:55.325217 16123 sys_catalog.cc:564] T 00000000000000000000000000000000 P a6d878afb46e4df18827ac19aa9f97bd [sys.catalog]: configured and running, proceeding with master startup.
W20260812 02:36:55.335840 16144 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a6d878afb46e4df18827ac19aa9f97bd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 02:36:55.336019 16144 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 02:36:55.339658 15940 master_runner.cc:428] Detected that this master a6d878afb46e4df18827ac19aa9f97bd is joining an existing cluster
I20260812 02:36:55.339761 15940 master_runner.cc:432] Initiating AddMaster RPC to add 127.7.132.188:39485
I20260812 02:36:55.368772 16041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:55.344128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39744 (local address 127.7.132.189:38815)
0812 02:36:55.344384 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:55.344393 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:55.346754 (+ 2361us) server_negotiation.cc:407] Connection header received
0812 02:36:55.347045 (+ 291us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:55.347054 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:55.347148 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:55.347231 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:55.348870 (+ 1639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:55.349939 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:55.355393 (+ 5454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:55.356119 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:55.356238 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:55.360458 (+ 4220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:55.360464 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:55.360469 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:55.360516 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:55.364123 (+ 3607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:55.365953 (+ 1830us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:55.365959 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:55.365964 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:55.366163 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:55.366858 (+ 695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:55.366864 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:55.366868 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:55.367279 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:55.367434 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:55.368130 (+ 696us) server_negotiation.cc:299] Negotiation successful
0812 02:36:55.368376 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260812 02:36:55.370029 16152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:55.343994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.189:38815 (local address 127.0.0.1:39744)
0812 02:36:55.346642 (+ 2648us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:55.346664 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:55.346876 (+ 212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:55.347490 (+ 614us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:55.347496 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:55.348258 (+ 762us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:36:55.348676 (+ 418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:55.348695 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:55.350179 (+ 1484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:55.350187 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:55.355175 (+ 4988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:55.355194 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:55.357748 (+ 2554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:55.357759 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:55.357972 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:55.358768 (+ 796us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:36:55.358791 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:36:55.360235 (+ 1444us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:36:55.365319 (+ 5084us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:55.365325 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:55.365330 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:55.365798 (+ 468us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:55.366338 (+ 540us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:55.366343 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:55.366348 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:55.366584 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:55.367520 (+ 936us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:36:55.367525 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:36:55.367756 (+ 231us) client_negotiation.cc:769] Sending connection context
0812 02:36:55.369058 (+ 1302us) client_negotiation.cc:240] Negotiation successful
0812 02:36:55.369479 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2457,"thread_start_us":2270,"threads_started":1}
I20260812 02:36:55.376077 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 15305
I20260812 02:36:55.383500 16153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:55.349682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:33723 (local address 127.0.0.1:42206)
0812 02:36:55.362566 (+ 12884us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:55.362588 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:55.362739 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:55.363390 (+ 651us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:55.363396 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:55.364095 (+ 699us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:36:55.364504 (+ 409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:55.364522 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:55.366009 (+ 1487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:55.366015 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:55.367981 (+ 1966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:55.367997 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:55.369260 (+ 1263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:55.369266 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:55.369471 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:55.370368 (+ 897us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:36:55.370391 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:36:55.373009 (+ 2618us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:36:55.379342 (+ 6333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:55.379348 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:55.379352 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:55.380668 (+ 1316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:55.381216 (+ 548us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:55.381221 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:55.381225 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:55.381458 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:55.382338 (+ 880us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:36:55.382343 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:36:55.382551 (+ 208us) client_negotiation.cc:769] Sending connection context
0812 02:36:55.382708 (+ 157us) client_negotiation.cc:240] Negotiation successful
0812 02:36:55.383031 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12730,"spinlock_wait_cycles":28928,"thread_start_us":13146,"threads_started":1}
I20260812 02:36:55.384765 16128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:55.349345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42206 (local address 127.7.132.190:33723)
0812 02:36:55.350910 (+ 1565us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:55.350916 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:55.362685 (+ 11769us) server_negotiation.cc:407] Connection header received
0812 02:36:55.362936 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:55.362944 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:55.363055 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:55.363182 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:55.364716 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:55.365790 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:55.368150 (+ 2360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:55.368830 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:55.368955 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:55.373384 (+ 4429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:55.373429 (+ 45us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:55.373435 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:55.373506 (+ 71us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:55.378150 (+ 4644us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:55.380824 (+ 2674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:55.380832 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:55.380835 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:55.381033 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:55.381615 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:55.381620 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:55.381626 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:55.382072 (+ 446us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:55.382248 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:55.383057 (+ 809us) server_negotiation.cc:299] Negotiation successful
0812 02:36:55.383524 (+ 467us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1376}
Aug 12 02:36:55 dist-test-slave-mn4r krb5kdc[15163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502211, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.132.129@KRBTEST.COM
I20260812 02:36:55.392805 16126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:55.363514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51758 (local address 127.7.132.188:39485)
0812 02:36:55.363718 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:55.363724 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:55.371048 (+ 7324us) server_negotiation.cc:407] Connection header received
0812 02:36:55.371293 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:55.371298 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:55.371382 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:55.371466 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:55.372841 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:55.374029 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:55.378390 (+ 4361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:55.379106 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:55.379231 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:55.385188 (+ 5957us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:55.385194 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:55.385199 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:55.385239 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:55.389560 (+ 4321us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:55.390421 (+ 861us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:55.390428 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:55.390433 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:55.390617 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:55.391080 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:55.391085 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:55.391088 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:55.391470 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:55.391619 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:55.392214 (+ 595us) server_negotiation.cc:299] Negotiation successful
0812 02:36:55.392438 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260812 02:36:55.399801 16152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:55.363851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:39485 (local address 127.0.0.1:51758)
0812 02:36:55.370957 (+ 7106us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:55.370978 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:55.371149 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:55.371689 (+ 540us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:55.371693 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:55.372372 (+ 679us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:36:55.372644 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:55.372665 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:55.374524 (+ 1859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:55.374529 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:55.376104 (+ 1575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:55.376122 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:55.379466 (+ 3344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:55.379472 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:55.379641 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:55.383806 (+ 4165us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:36:55.383828 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:36:55.384999 (+ 1171us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:36:55.389720 (+ 4721us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:55.389725 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:55.389731 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:55.390287 (+ 556us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:55.390764 (+ 477us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:55.390769 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:55.390772 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:55.390956 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:55.391698 (+ 742us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:36:55.391702 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:36:55.391868 (+ 166us) client_negotiation.cc:769] Sending connection context
0812 02:36:55.398904 (+ 7036us) client_negotiation.cc:240] Negotiation successful
0812 02:36:55.399270 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6981,"thread_start_us":7583,"threads_started":1}
Aug 12 02:36:55 dist-test-slave-mn4r krb5kdc[15163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502212, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.132.190@KRBTEST.COM for oryx/127.7.132.188@KRBTEST.COM
I20260812 02:36:55.405633 16155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:55.378659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51396 (local address 127.7.132.129:34169)
0812 02:36:55.379397 (+ 738us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:55.379407 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:55.379439 (+ 32us) server_negotiation.cc:407] Connection header received
0812 02:36:55.379514 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:55.379523 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:55.379866 (+ 343us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:55.380052 (+ 186us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:55.381788 (+ 1736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:55.383485 (+ 1697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:55.385534 (+ 2049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:55.386640 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:55.387370 (+ 730us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:55.392703 (+ 5333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:55.392718 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:55.392739 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:55.392782 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:55.399374 (+ 6592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:55.400343 (+ 969us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:55.400353 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:55.400357 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:55.401833 (+ 1476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:55.402498 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:55.402503 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:55.402506 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:55.403102 (+ 596us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:55.404706 (+ 1604us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:55.404899 (+ 193us) server_negotiation.cc:299] Negotiation successful
0812 02:36:55.405176 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":553,"thread_start_us":392,"threads_started":1}
I20260812 02:36:55.407473 15454 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:36:55.416939 15923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:55.391475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:39485 (local address 127.0.0.1:51762)
0812 02:36:55.391846 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:55.391866 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:55.391972 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:55.393481 (+ 1509us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:55.393489 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:55.394012 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:36:55.394272 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:55.394287 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:55.396202 (+ 1915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:55.396210 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:55.397716 (+ 1506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:55.397734 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:55.400557 (+ 2823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:55.400568 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:55.400823 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:55.401472 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:36:55.401495 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:36:55.404850 (+ 3355us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:36:55.409759 (+ 4909us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:55.409776 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:55.409780 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:55.410359 (+ 579us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:55.410814 (+ 455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:55.410832 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:55.410836 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:55.411018 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:55.415815 (+ 4797us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:36:55.415821 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:36:55.416051 (+ 230us) client_negotiation.cc:769] Sending connection context
0812 02:36:55.416215 (+ 164us) client_negotiation.cc:240] Negotiation successful
0812 02:36:55.416532 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240}
I20260812 02:36:55.418243 16126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:55.392180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51762 (local address 127.7.132.188:39485)
0812 02:36:55.393058 (+ 878us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:55.393063 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:55.393079 (+ 16us) server_negotiation.cc:407] Connection header received
0812 02:36:55.393171 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:55.393179 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:55.393298 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:55.394434 (+ 1136us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:55.394618 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:55.396015 (+ 1397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:55.397997 (+ 1982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:55.398773 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:55.401144 (+ 2371us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:55.405073 (+ 3929us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:55.405079 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:55.405084 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:55.405119 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:55.409520 (+ 4401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:55.410533 (+ 1013us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:55.410539 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:55.410543 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:55.410679 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:55.411174 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:55.411179 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:55.411182 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:55.415648 (+ 4466us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:55.417387 (+ 1739us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:55.417517 (+ 130us) server_negotiation.cc:299] Negotiation successful
0812 02:36:55.417763 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":790,"thread_start_us":5262,"threads_started":1}
I20260812 02:36:55.419759 15223 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.7.132.188:39485: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "a6d878afb46e4df18827ac19aa9f97bd"
member_type: NON_VOTER
last_known_addr {
host: "127.7.132.188"
port: 39485
}
attrs {
promote: true
}
}
dest_uuid: "82e54874d347460c85c36da26b3bb8c8"
cas_config_opid_index: 6
I20260812 02:36:55.420648 15223 consensus_queue.cc:237] T 00000000000000000000000000000000 P 82e54874d347460c85c36da26b3bb8c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "82e54874d347460c85c36da26b3bb8c8" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 33723 } } peers { permanent_uuid: "351974fae6dd4c168a844a01b53d9927" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 38815 } attrs { promote: false } } peers { permanent_uuid: "a6d878afb46e4df18827ac19aa9f97bd" member_type: NON_VOTER last_known_addr { host: "127.7.132.188" port: 39485 } attrs { promote: true } }
I20260812 02:36:55.423647 15998 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 351974fae6dd4c168a844a01b53d9927 [term 1 FOLLOWER]: Refusing update from remote peer 82e54874d347460c85c36da26b3bb8c8: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260812 02:36:55.424530 15930 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 82e54874d347460c85c36da26b3bb8c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "351974fae6dd4c168a844a01b53d9927" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 38815 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260812 02:36:55.427276 16091 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a6d878afb46e4df18827ac19aa9f97bd [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:36:55.432538 16091 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a6d878afb46e4df18827ac19aa9f97bd [term 1 FOLLOWER]: Refusing update from remote peer 82e54874d347460c85c36da26b3bb8c8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260812 02:36:55.432454 15930 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 82e54874d347460c85c36da26b3bb8c8 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER a6d878afb46e4df18827ac19aa9f97bd (127.7.132.188) added. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "82e54874d347460c85c36da26b3bb8c8" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 33723 } } peers { permanent_uuid: "351974fae6dd4c168a844a01b53d9927" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 38815 } attrs { promote: false } } peers { permanent_uuid: "a6d878afb46e4df18827ac19aa9f97bd" member_type: NON_VOTER last_known_addr { host: "127.7.132.188" port: 39485 } attrs { promote: true } } }
I20260812 02:36:55.436740 15930 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.7.132.188:39485
I20260812 02:36:55.437016 15930 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 82e54874d347460c85c36da26b3bb8c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6d878afb46e4df18827ac19aa9f97bd" member_type: NON_VOTER last_known_addr { host: "127.7.132.188" port: 39485 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:36:55.438293 15929 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82e54874d347460c85c36da26b3bb8c8 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "82e54874d347460c85c36da26b3bb8c8" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "82e54874d347460c85c36da26b3bb8c8" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 33723 } } peers { permanent_uuid: "351974fae6dd4c168a844a01b53d9927" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 38815 } attrs { promote: false } } peers { permanent_uuid: "a6d878afb46e4df18827ac19aa9f97bd" member_type: NON_VOTER last_known_addr { host: "127.7.132.188" port: 39485 } attrs { promote: true } } }
I20260812 02:36:55.438562 15929 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82e54874d347460c85c36da26b3bb8c8 [sys.catalog]: This master's current role is: LEADER
I20260812 02:36:55.438819 15929 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82e54874d347460c85c36da26b3bb8c8 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "82e54874d347460c85c36da26b3bb8c8" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "82e54874d347460c85c36da26b3bb8c8" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 33723 } } peers { permanent_uuid: "351974fae6dd4c168a844a01b53d9927" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 38815 } attrs { promote: false } } peers { permanent_uuid: "a6d878afb46e4df18827ac19aa9f97bd" member_type: NON_VOTER last_known_addr { host: "127.7.132.188" port: 39485 } attrs { promote: true } } }
I20260812 02:36:55.439036 15929 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82e54874d347460c85c36da26b3bb8c8 [sys.catalog]: This master's current role is: LEADER
I20260812 02:36:55.439574 15998 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 351974fae6dd4c168a844a01b53d9927 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER a6d878afb46e4df18827ac19aa9f97bd (127.7.132.188) added. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "82e54874d347460c85c36da26b3bb8c8" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 33723 } } peers { permanent_uuid: "351974fae6dd4c168a844a01b53d9927" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 38815 } attrs { promote: false } } peers { permanent_uuid: "a6d878afb46e4df18827ac19aa9f97bd" member_type: NON_VOTER last_known_addr { host: "127.7.132.188" port: 39485 } attrs { promote: true } } }
I20260812 02:36:55.442824 15940 master.cc:537] Master@127.7.132.188:39485 shutting down...
I20260812 02:36:55.460543 16027 sys_catalog.cc:455] T 00000000000000000000000000000000 P 351974fae6dd4c168a844a01b53d9927 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "82e54874d347460c85c36da26b3bb8c8" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "82e54874d347460c85c36da26b3bb8c8" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 33723 } } peers { permanent_uuid: "351974fae6dd4c168a844a01b53d9927" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 38815 } attrs { promote: false } } peers { permanent_uuid: "a6d878afb46e4df18827ac19aa9f97bd" member_type: NON_VOTER last_known_addr { host: "127.7.132.188" port: 39485 } attrs { promote: true } } }
I20260812 02:36:55.460867 16027 sys_catalog.cc:458] T 00000000000000000000000000000000 P 351974fae6dd4c168a844a01b53d9927 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 02:36:55.463244 16091 raft_consensus.cc:858] T 00000000000000000000000000000000 P a6d878afb46e4df18827ac19aa9f97bd [term 1 NON_PARTICIPANT]: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82e54874d347460c85c36da26b3bb8c8" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 33723 } } peers { permanent_uuid: "351974fae6dd4c168a844a01b53d9927" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 38815 } } peers { permanent_uuid: "a6d878afb46e4df18827ac19aa9f97bd" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 39485 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "82e54874d347460c85c36da26b3bb8c8" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 33723 } } peers { permanent_uuid: "351974fae6dd4c168a844a01b53d9927" member_type: NON_VOTER last_known_addr { host: "127.7.132.189" port: 38815 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "82e54874d347460c85c36da26b3bb8c8" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 33723 } } peers { permanent_uuid: "351974fae6dd4c168a844a01b53d9927" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 38815 } attrs { promote: false } }
W20260812 02:36:55.463451 16091 raft_consensus.cc:1565] T 00000000000000000000000000000000 P a6d878afb46e4df18827ac19aa9f97bd [term 1 NON_PARTICIPANT]: Could not prepare op 'term: 1 index: 6' and following 1 ops. Status for this op: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82e54874d347460c85c36da26b3bb8c8" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 33723 } } peers { permanent_uuid: "351974fae6dd4c168a844a01b53d9927" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 38815 } } peers { permanent_uuid: "a6d878afb46e4df18827ac19aa9f97bd" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 39485 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "82e54874d347460c85c36da26b3bb8c8" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 33723 } } peers { permanent_uuid: "351974fae6dd4c168a844a01b53d9927" member_type: NON_VOTER last_known_addr { host: "127.7.132.189" port: 38815 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "82e54874d347460c85c36da26b3bb8c8" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 33723 } } peers { permanent_uuid: "351974fae6dd4c168a844a01b53d9927" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 38815 } attrs { promote: false } }
W20260812 02:36:55.486408 16067 tablet.cc:2369] T 00000000000000000000000000000000 P a6d878afb46e4df18827ac19aa9f97bd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 02:36:55.494907 16091 raft_consensus.cc:2932] T 00000000000000000000000000000000 P a6d878afb46e4df18827ac19aa9f97bd [term 1 NON_PARTICIPANT]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 351974fae6dd4c168a844a01b53d9927 (127.7.132.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "82e54874d347460c85c36da26b3bb8c8" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 33723 } } peers { permanent_uuid: "351974fae6dd4c168a844a01b53d9927" member_type: NON_VOTER last_known_addr { host: "127.7.132.189" port: 38815 } attrs { promote: true } } }
I20260812 02:36:55.500829 16130 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6d878afb46e4df18827ac19aa9f97bd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 82e54874d347460c85c36da26b3bb8c8. Latest consensus state: current_term: 1 leader_uuid: "82e54874d347460c85c36da26b3bb8c8" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "82e54874d347460c85c36da26b3bb8c8" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 33723 } } peers { permanent_uuid: "351974fae6dd4c168a844a01b53d9927" member_type: NON_VOTER last_known_addr { host: "127.7.132.189" port: 38815 } attrs { promote: true } } }
I20260812 02:36:55.501062 16130 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6d878afb46e4df18827ac19aa9f97bd [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260812 02:36:55.501247 16130 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6d878afb46e4df18827ac19aa9f97bd [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "82e54874d347460c85c36da26b3bb8c8" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "82e54874d347460c85c36da26b3bb8c8" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 33723 } } peers { permanent_uuid: "351974fae6dd4c168a844a01b53d9927" member_type: NON_VOTER last_known_addr { host: "127.7.132.189" port: 38815 } attrs { promote: true } } }
I20260812 02:36:55.501415 16130 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6d878afb46e4df18827ac19aa9f97bd [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260812 02:36:55.501327 15928 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82e54874d347460c85c36da26b3bb8c8 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "82e54874d347460c85c36da26b3bb8c8" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "82e54874d347460c85c36da26b3bb8c8" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 33723 } } peers { permanent_uuid: "351974fae6dd4c168a844a01b53d9927" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 38815 } attrs { promote: false } } peers { permanent_uuid: "a6d878afb46e4df18827ac19aa9f97bd" member_type: NON_VOTER last_known_addr { host: "127.7.132.188" port: 39485 } attrs { promote: true } } }
I20260812 02:36:55.501670 15928 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82e54874d347460c85c36da26b3bb8c8 [sys.catalog]: This master's current role is: LEADER
I20260812 02:36:55.500164 16164 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6d878afb46e4df18827ac19aa9f97bd [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "82e54874d347460c85c36da26b3bb8c8" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "82e54874d347460c85c36da26b3bb8c8" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 33723 } } peers { permanent_uuid: "351974fae6dd4c168a844a01b53d9927" member_type: NON_VOTER last_known_addr { host: "127.7.132.189" port: 38815 } attrs { promote: true } } }
I20260812 02:36:55.501762 16164 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6d878afb46e4df18827ac19aa9f97bd [sys.catalog]: This master's current role is: NON_PARTICIPANT
W20260812 02:36:55.503207 15205 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260812 02:36:55.504815 15940 raft_consensus.cc:2229] T 00000000000000000000000000000000 P a6d878afb46e4df18827ac19aa9f97bd [term 1 NON_PARTICIPANT]: Raft consensus shutting down.
I20260812 02:36:55.505280 15940 raft_consensus.cc:2258] T 00000000000000000000000000000000 P a6d878afb46e4df18827ac19aa9f97bd [term 1 NON_PARTICIPANT]: Raft consensus is shut down!
I20260812 02:36:55.505468 15940 tablet_replica.cc:330] T 00000000000000000000000000000000 P a6d878afb46e4df18827ac19aa9f97bd: stopping tablet replica
W20260812 02:36:55.505620 15205 consensus_peers.cc:487] T 00000000000000000000000000000000 P 82e54874d347460c85c36da26b3bb8c8 -> Peer a6d878afb46e4df18827ac19aa9f97bd (127.7.132.188:39485): Couldn't send request to peer a6d878afb46e4df18827ac19aa9f97bd. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260812 02:36:55.523461 15940 master.cc:552] Master@127.7.132.188:39485 shutdown complete.
I20260812 02:36:55.523583 15940 master_runner.cc:305] Clearing existing system tablet
I20260812 02:36:55.523936 15940 fs_manager.cc:359] Metadata directory not provided
I20260812 02:36:55.524080 15940 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:36:55.533806 15940 fs_manager.cc:709] Time spent opening directory manager: real 0.009s user 0.002s sys 0.009s
I20260812 02:36:55.535578 16170 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:36:55.535948 15940 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:36:55.536242 15940 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260812 02:36:55.536350 15940 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/master-2/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/master-2/wal
uuid: "a6d878afb46e4df18827ac19aa9f97bd"
format_stamp: "Formatted at 2026-08-12 02:36:55 on dist-test-slave-mn4r"
server_key: "c4f21b60af8c39038bb7cb8311d650ce"
server_key_iv: "b199d2cc1a0a458e39ae94a9d9e4eb03"
server_key_version: "encryptionkey@0"
I20260812 02:36:55.540127 15940 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P a6d878afb46e4df18827ac19aa9f97bd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:36:55.546200 15940 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P a6d878afb46e4df18827ac19aa9f97bd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260812 02:36:55.546391 15940 log.cc:1196] T 00000000000000000000000000000000 P a6d878afb46e4df18827ac19aa9f97bd: Deleting WAL directory at /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260812 02:36:55.546846 15940 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P a6d878afb46e4df18827ac19aa9f97bd: Deleting consensus metadata
I20260812 02:36:55.547129 15940 master_runner.cc:315] Copying system tablet from 127.7.132.190:33723
I20260812 02:36:55.549945 15940 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P a6d878afb46e4df18827ac19aa9f97bd: tablet copy: Beginning tablet copy session from remote peer at address 127.7.132.190:33723
I20260812 02:36:55.550136 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 15305
I20260812 02:36:55.578989 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.129:34169
--local_ip_for_outbound_sockets=127.7.132.129
--tserver_master_addrs=127.7.132.190:33723
--webserver_port=44947
--webserver_interface=127.7.132.129
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/krb5kdc/oryx_127.7.132.129.keytab
--principal=oryx/127.7.132.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:43909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.7.132.190:33723,127.7.132.189:38815
--tserver_master_addrs=127.7.132.190:33723,127.7.132.189:38815,127.7.132.188:39485 with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 02:36:55.581938 16178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:55.554871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:33723 (local address 127.0.0.1:42210)
0812 02:36:55.566223 (+ 11352us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:55.566254 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:55.566508 (+ 254us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:55.567102 (+ 594us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:55.567107 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:55.567687 (+ 580us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:36:55.567952 (+ 265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:55.567965 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:55.569550 (+ 1585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:55.569555 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:55.571160 (+ 1605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:55.571175 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:55.572392 (+ 1217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:55.572400 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:55.572614 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:55.573357 (+ 743us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:36:55.573381 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:36:55.574575 (+ 1194us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:36:55.578995 (+ 4420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:55.579001 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:55.579005 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:55.579540 (+ 535us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:55.579996 (+ 456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:55.580001 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:55.580005 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:55.580156 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:55.580718 (+ 562us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:36:55.580723 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:36:55.580937 (+ 214us) client_negotiation.cc:769] Sending connection context
0812 02:36:55.581087 (+ 150us) client_negotiation.cc:240] Negotiation successful
0812 02:36:55.581432 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11172,"thread_start_us":11006,"threads_started":1}
I20260812 02:36:55.581938 16128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:55.554364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42210 (local address 127.7.132.190:33723)
0812 02:36:55.554641 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:55.554650 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:55.566373 (+ 11723us) server_negotiation.cc:407] Connection header received
0812 02:36:55.566677 (+ 304us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:55.566685 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:55.566786 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:55.566865 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:55.568159 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:55.569347 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:55.571308 (+ 1961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:55.572178 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:55.572338 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:55.574806 (+ 2468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:55.574816 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:55.574824 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:55.574913 (+ 89us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:55.578844 (+ 3931us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:55.579698 (+ 854us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:55.579704 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:55.579707 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:55.579854 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:55.580278 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:55.580283 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:55.580286 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:55.580590 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:55.580780 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:55.581218 (+ 438us) server_negotiation.cc:299] Negotiation successful
0812 02:36:55.581523 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129}
I20260812 02:36:55.583108 15246 tablet_copy_service.cc:140] P 82e54874d347460c85c36da26b3bb8c8: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer a6d878afb46e4df18827ac19aa9f97bd ({username='oryx', principal='oryx/127.7.132.188@KRBTEST.COM'} at 127.0.0.1:42210)
I20260812 02:36:55.583290 15246 tablet_copy_service.cc:161] P 82e54874d347460c85c36da26b3bb8c8: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer a6d878afb46e4df18827ac19aa9f97bd at {username='oryx', principal='oryx/127.7.132.188@KRBTEST.COM'} at 127.0.0.1:42210: session id = a6d878afb46e4df18827ac19aa9f97bd-00000000000000000000000000000000
I20260812 02:36:55.585611 15246 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 82e54874d347460c85c36da26b3bb8c8: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 02:36:55.587298 15940 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260812 02:36:55.592918 15940 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P a6d878afb46e4df18827ac19aa9f97bd: tablet copy: Starting download of 0 data blocks...
I20260812 02:36:55.593227 15940 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P a6d878afb46e4df18827ac19aa9f97bd: tablet copy: Starting download of 1 WAL segments...
I20260812 02:36:55.595563 15940 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P a6d878afb46e4df18827ac19aa9f97bd: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 02:36:55.599086 15246 tablet_copy_service.cc:342] P 82e54874d347460c85c36da26b3bb8c8: Request end of tablet copy session a6d878afb46e4df18827ac19aa9f97bd-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.7.132.188@KRBTEST.COM'} at 127.0.0.1:42210
I20260812 02:36:55.599227 15246 tablet_copy_service.cc:433] P 82e54874d347460c85c36da26b3bb8c8: ending tablet copy session a6d878afb46e4df18827ac19aa9f97bd-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer a6d878afb46e4df18827ac19aa9f97bd
I20260812 02:36:55.613627 15940 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:36:55.619953 16180 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:36:55.620424 16181 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:36:55.624104 15940 server_base.cc:1034] running on GCE node
W20260812 02:36:55.624776 16183 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:36:55.625347 15940 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:36:55.626010 15940 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:36:55.627269 15940 hybrid_clock.cc:648] HybridClock initialized: now 1786502215627224 us; error 62 us; skew 500 ppm
I20260812 02:36:55.627550 15940 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:36:55 dist-test-slave-mn4r krb5kdc[15163](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502215, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.132.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:36:55.629845 15940 init.cc:377] Logged in from keytab as oryx/127.7.132.188@KRBTEST.COM (short username oryx)
I20260812 02:36:55.635083 15940 webserver.cc:460] Webserver started at http://127.7.132.188:36791/ using document root <none> and password file <none>
I20260812 02:36:55.635442 15940 fs_manager.cc:359] Metadata directory not provided
I20260812 02:36:55.635556 15940 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:36:55.638696 15940 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.003s
I20260812 02:36:55.640709 16191 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:36:55.641275 15940 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260812 02:36:55.641417 15940 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/master-2/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/master-2/wal
uuid: "a6d878afb46e4df18827ac19aa9f97bd"
format_stamp: "Formatted at 2026-08-12 02:36:55 on dist-test-slave-mn4r"
server_key: "c4f21b60af8c39038bb7cb8311d650ce"
server_key_iv: "b199d2cc1a0a458e39ae94a9d9e4eb03"
server_key_version: "encryptionkey@0"
I20260812 02:36:55.641616 15940 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:36:55.658118 15940 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:36:55.662649 15940 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:36:55.690374 15940 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.188:39485
I20260812 02:36:55.690439 16258 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.188:39485 every 8 connection(s)
I20260812 02:36:55.691653 15940 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/master-2/data/info.pb
I20260812 02:36:55.697469 16259 sys_catalog.cc:263] Verifying existing consensus state
I20260812 02:36:55.701617 16259 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6d878afb46e4df18827ac19aa9f97bd: Bootstrap starting.
I20260812 02:36:55.717737 16259 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6d878afb46e4df18827ac19aa9f97bd: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 02:36:55.718360 16259 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6d878afb46e4df18827ac19aa9f97bd: Bootstrap complete.
I20260812 02:36:55.719614 16259 raft_consensus.cc:348] T 00000000000000000000000000000000 P a6d878afb46e4df18827ac19aa9f97bd [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "82e54874d347460c85c36da26b3bb8c8" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 33723 } } peers { permanent_uuid: "351974fae6dd4c168a844a01b53d9927" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 38815 } attrs { promote: false } } peers { permanent_uuid: "a6d878afb46e4df18827ac19aa9f97bd" member_type: NON_VOTER last_known_addr { host: "127.7.132.188" port: 39485 } attrs { promote: true } }
I20260812 02:36:55.719904 16259 raft_consensus.cc:729] T 00000000000000000000000000000000 P a6d878afb46e4df18827ac19aa9f97bd [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a6d878afb46e4df18827ac19aa9f97bd, State: Initialized, Role: LEARNER
I20260812 02:36:55.720171 16259 consensus_queue.cc:260] T 00000000000000000000000000000000 P a6d878afb46e4df18827ac19aa9f97bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "82e54874d347460c85c36da26b3bb8c8" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 33723 } } peers { permanent_uuid: "351974fae6dd4c168a844a01b53d9927" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 38815 } attrs { promote: false } } peers { permanent_uuid: "a6d878afb46e4df18827ac19aa9f97bd" member_type: NON_VOTER last_known_addr { host: "127.7.132.188" port: 39485 } attrs { promote: true } }
I20260812 02:36:55.721381 16259 sys_catalog.cc:564] T 00000000000000000000000000000000 P a6d878afb46e4df18827ac19aa9f97bd [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:36:55.721103 16262 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6d878afb46e4df18827ac19aa9f97bd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "82e54874d347460c85c36da26b3bb8c8" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 33723 } } peers { permanent_uuid: "351974fae6dd4c168a844a01b53d9927" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 38815 } attrs { promote: false } } peers { permanent_uuid: "a6d878afb46e4df18827ac19aa9f97bd" member_type: NON_VOTER last_known_addr { host: "127.7.132.188" port: 39485 } attrs { promote: true } } }
I20260812 02:36:55.722033 16262 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6d878afb46e4df18827ac19aa9f97bd [sys.catalog]: This master's current role is: LEARNER
I20260812 02:36:55.729544 16273 catalog_manager.cc:1259] Loaded cluster ID: eb5dad367dc04d1ba16a1cfb4c2dc8a7
I20260812 02:36:55.729707 16273 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a6d878afb46e4df18827ac19aa9f97bd: loading cluster ID for follower catalog manager: success
I20260812 02:36:55.733278 16273 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a6d878afb46e4df18827ac19aa9f97bd: acquiring CA information for follower catalog manager: success
I20260812 02:36:55.735858 16273 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a6d878afb46e4df18827ac19aa9f97bd: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 02:36:55.938182 16159 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82e54874d347460c85c36da26b3bb8c8 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "82e54874d347460c85c36da26b3bb8c8" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "82e54874d347460c85c36da26b3bb8c8" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 33723 } } peers { permanent_uuid: "351974fae6dd4c168a844a01b53d9927" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 38815 } attrs { promote: false } } peers { permanent_uuid: "a6d878afb46e4df18827ac19aa9f97bd" member_type: NON_VOTER last_known_addr { host: "127.7.132.188" port: 39485 } attrs { promote: true } } }
I20260812 02:36:55.938593 16159 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82e54874d347460c85c36da26b3bb8c8 [sys.catalog]: This master's current role is: LEADER
I20260812 02:36:55.947017 16274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:55.938517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:39485 (local address 127.0.0.1:51766)
0812 02:36:55.940008 (+ 1491us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:55.940038 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:55.940191 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:55.940670 (+ 479us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:55.940677 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:55.940708 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:36:55.940948 (+ 240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:55.940959 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:55.942980 (+ 2021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:55.942986 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:55.944540 (+ 1554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:55.944556 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:55.946406 (+ 1850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:55.946411 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:55.946556 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:55.946583 (+ 27us) client_negotiation.cc:769] Sending connection context
0812 02:36:55.946652 (+ 69us) client_negotiation.cc:240] Negotiation successful
0812 02:36:55.946751 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1368,"thread_start_us":1280,"threads_started":1}
I20260812 02:36:55.947217 16275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:55.938692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51766 (local address 127.7.132.188:39485)
0812 02:36:55.939641 (+ 949us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:55.939653 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:55.940122 (+ 469us) server_negotiation.cc:407] Connection header received
0812 02:36:55.940344 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:55.940349 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:55.940470 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:55.940617 (+ 147us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:36:55.941112 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:55.942774 (+ 1662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:55.944729 (+ 1955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:55.946200 (+ 1471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:55.946352 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:55.946645 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:55.946707 (+ 62us) server_negotiation.cc:299] Negotiation successful
0812 02:36:55.946873 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":721,"thread_start_us":859,"threads_started":1}
I20260812 02:36:55.948138 16228 raft_consensus.cc:1203] T 00000000000000000000000000000000 P a6d878afb46e4df18827ac19aa9f97bd [term 1 LEARNER]: Deduplicated request from leader. Original: 1.5->[1.6-1.7] Dedup: 1.7->[]
I20260812 02:36:55.948601 16262 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6d878afb46e4df18827ac19aa9f97bd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 82e54874d347460c85c36da26b3bb8c8. Latest consensus state: current_term: 1 leader_uuid: "82e54874d347460c85c36da26b3bb8c8" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "82e54874d347460c85c36da26b3bb8c8" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 33723 } } peers { permanent_uuid: "351974fae6dd4c168a844a01b53d9927" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 38815 } attrs { promote: false } } peers { permanent_uuid: "a6d878afb46e4df18827ac19aa9f97bd" member_type: NON_VOTER last_known_addr { host: "127.7.132.188" port: 39485 } attrs { promote: true } } }
I20260812 02:36:55.948925 16262 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6d878afb46e4df18827ac19aa9f97bd [sys.catalog]: This master's current role is: LEARNER
I20260812 02:36:55.949275 16159 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 82e54874d347460c85c36da26b3bb8c8: attempting to promote NON_VOTER a6d878afb46e4df18827ac19aa9f97bd to VOTER
I20260812 02:36:55.949891 16159 consensus_queue.cc:237] T 00000000000000000000000000000000 P 82e54874d347460c85c36da26b3bb8c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "82e54874d347460c85c36da26b3bb8c8" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 33723 } } peers { permanent_uuid: "351974fae6dd4c168a844a01b53d9927" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 38815 } attrs { promote: false } } peers { permanent_uuid: "a6d878afb46e4df18827ac19aa9f97bd" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 39485 } attrs { promote: false } }
I20260812 02:36:55.952069 15998 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 351974fae6dd4c168a844a01b53d9927 [term 1 FOLLOWER]: Refusing update from remote peer 82e54874d347460c85c36da26b3bb8c8: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260812 02:36:55.952687 16228 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a6d878afb46e4df18827ac19aa9f97bd [term 1 LEARNER]: Refusing update from remote peer 82e54874d347460c85c36da26b3bb8c8: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260812 02:36:55.952744 15930 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 82e54874d347460c85c36da26b3bb8c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "351974fae6dd4c168a844a01b53d9927" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 38815 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260812 02:36:55.953205 15928 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 82e54874d347460c85c36da26b3bb8c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6d878afb46e4df18827ac19aa9f97bd" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 39485 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260812 02:36:55.955689 15928 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 82e54874d347460c85c36da26b3bb8c8 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, a6d878afb46e4df18827ac19aa9f97bd (127.7.132.188) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "82e54874d347460c85c36da26b3bb8c8" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 33723 } } peers { permanent_uuid: "351974fae6dd4c168a844a01b53d9927" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 38815 } attrs { promote: false } } peers { permanent_uuid: "a6d878afb46e4df18827ac19aa9f97bd" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 39485 } attrs { promote: false } } }
I20260812 02:36:55.959864 15928 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82e54874d347460c85c36da26b3bb8c8 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "82e54874d347460c85c36da26b3bb8c8" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "82e54874d347460c85c36da26b3bb8c8" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 33723 } } peers { permanent_uuid: "351974fae6dd4c168a844a01b53d9927" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 38815 } attrs { promote: false } } peers { permanent_uuid: "a6d878afb46e4df18827ac19aa9f97bd" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 39485 } attrs { promote: false } } }
I20260812 02:36:55.960148 15928 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82e54874d347460c85c36da26b3bb8c8 [sys.catalog]: This master's current role is: LEADER
I20260812 02:36:55.960415 16159 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82e54874d347460c85c36da26b3bb8c8 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "82e54874d347460c85c36da26b3bb8c8" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "82e54874d347460c85c36da26b3bb8c8" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 33723 } } peers { permanent_uuid: "351974fae6dd4c168a844a01b53d9927" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 38815 } attrs { promote: false } } peers { permanent_uuid: "a6d878afb46e4df18827ac19aa9f97bd" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 39485 } attrs { promote: false } } }
I20260812 02:36:55.960675 16159 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82e54874d347460c85c36da26b3bb8c8 [sys.catalog]: This master's current role is: LEADER
I20260812 02:36:55.960487 16228 raft_consensus.cc:2932] T 00000000000000000000000000000000 P a6d878afb46e4df18827ac19aa9f97bd [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, a6d878afb46e4df18827ac19aa9f97bd (127.7.132.188) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "82e54874d347460c85c36da26b3bb8c8" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 33723 } } peers { permanent_uuid: "351974fae6dd4c168a844a01b53d9927" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 38815 } attrs { promote: false } } peers { permanent_uuid: "a6d878afb46e4df18827ac19aa9f97bd" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 39485 } attrs { promote: false } } }
I20260812 02:36:55.961635 15998 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 351974fae6dd4c168a844a01b53d9927 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, a6d878afb46e4df18827ac19aa9f97bd (127.7.132.188) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "82e54874d347460c85c36da26b3bb8c8" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 33723 } } peers { permanent_uuid: "351974fae6dd4c168a844a01b53d9927" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 38815 } attrs { promote: false } } peers { permanent_uuid: "a6d878afb46e4df18827ac19aa9f97bd" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 39485 } attrs { promote: false } } }
I20260812 02:36:55.962029 16159 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82e54874d347460c85c36da26b3bb8c8 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "82e54874d347460c85c36da26b3bb8c8" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "82e54874d347460c85c36da26b3bb8c8" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 33723 } } peers { permanent_uuid: "351974fae6dd4c168a844a01b53d9927" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 38815 } attrs { promote: false } } peers { permanent_uuid: "a6d878afb46e4df18827ac19aa9f97bd" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 39485 } attrs { promote: false } } }
I20260812 02:36:55.962265 16159 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82e54874d347460c85c36da26b3bb8c8 [sys.catalog]: This master's current role is: LEADER
I20260812 02:36:55.964545 16262 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6d878afb46e4df18827ac19aa9f97bd [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "82e54874d347460c85c36da26b3bb8c8" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "82e54874d347460c85c36da26b3bb8c8" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 33723 } } peers { permanent_uuid: "351974fae6dd4c168a844a01b53d9927" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 38815 } attrs { promote: false } } peers { permanent_uuid: "a6d878afb46e4df18827ac19aa9f97bd" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 39485 } attrs { promote: false } } }
I20260812 02:36:55.964766 16262 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6d878afb46e4df18827ac19aa9f97bd [sys.catalog]: This master's current role is: FOLLOWER
I20260812 02:36:55.966110 16279 sys_catalog.cc:455] T 00000000000000000000000000000000 P 351974fae6dd4c168a844a01b53d9927 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "82e54874d347460c85c36da26b3bb8c8" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "82e54874d347460c85c36da26b3bb8c8" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 33723 } } peers { permanent_uuid: "351974fae6dd4c168a844a01b53d9927" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 38815 } attrs { promote: false } } peers { permanent_uuid: "a6d878afb46e4df18827ac19aa9f97bd" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 39485 } attrs { promote: false } } }
I20260812 02:36:55.966389 16279 sys_catalog.cc:458] T 00000000000000000000000000000000 P 351974fae6dd4c168a844a01b53d9927 [sys.catalog]: This master's current role is: FOLLOWER
W20260812 02:36:55.985741 16179 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:36:55.986024 16179 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:36:55.986203 16179 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:36:55.993355 16179 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:36:55.993484 16179 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:36:55.993615 16179 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.129
I20260812 02:36:56.004020 16179 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:43909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/krb5kdc/oryx_127.7.132.129.keytab
--principal=oryx/127.7.132.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.129:34169
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.132.129
--webserver_port=44947
--tserver_master_addrs=127.7.132.190:33723,127.7.132.189:38815,127.7.132.188:39485
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.129
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:36:56.005383 16179 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:36:56.006896 16179 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:36:56.021078 16285 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:36:56.021150 16288 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:36:56.021126 16286 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:36:56.021994 16179 server_base.cc:1034] running on GCE node
I20260812 02:36:56.022635 16179 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:36:56.023555 16179 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:36:56.024794 16179 hybrid_clock.cc:648] HybridClock initialized: now 1786502216024735 us; error 72 us; skew 500 ppm
I20260812 02:36:56.025190 16179 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:36:56 dist-test-slave-mn4r krb5kdc[15163](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502216, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.132.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:36:56.028570 16179 init.cc:377] Logged in from keytab as oryx/127.7.132.129@KRBTEST.COM (short username oryx)
I20260812 02:36:56.030277 16179 webserver.cc:460] Webserver started at http://127.7.132.129:44947/ using document root <none> and password file <none>
I20260812 02:36:56.031018 16179 fs_manager.cc:359] Metadata directory not provided
I20260812 02:36:56.031183 16179 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:36:56.036666 16179 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.003s
I20260812 02:36:56.039477 16295 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:36:56.040915 16179 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 02:36:56.041107 16179 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/ts-0/wal
uuid: "ed2e8f19bfd642ac95e6b32d054ce8f4"
format_stamp: "Formatted at 2026-08-12 02:36:52 on dist-test-slave-mn4r"
server_key: "ac74a91345f24659f567325fd15c4afd"
server_key_iv: "bd41d6e3f396cbbdbd2b95b8d68ee332"
server_key_version: "encryptionkey@0"
I20260812 02:36:56.042295 16179 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:36:56.077936 16179 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:36:56.082635 16179 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:36:56.082890 16179 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:36:56.083940 16179 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:36:56.085299 16179 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:36:56.085404 16179 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:36:56.085493 16179 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:36:56.085551 16179 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:36:56.122412 16179 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.129:34169
I20260812 02:36:56.122478 16433 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.129:34169 every 8 connection(s)
I20260812 02:36:56.123876 16179 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/ts-0/data/info.pb
I20260812 02:36:56.131515 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 16179
I20260812 02:36:56.131959 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 15466
Aug 12 02:36:56 dist-test-slave-mn4r krb5kdc[15163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502211, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.132.130@KRBTEST.COM
Aug 12 02:36:56 dist-test-slave-mn4r krb5kdc[15163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502216, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.132.129@KRBTEST.COM for oryx/127.7.132.188@KRBTEST.COM
Aug 12 02:36:56 dist-test-slave-mn4r krb5kdc[15163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502216, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.132.129@KRBTEST.COM for oryx/127.7.132.189@KRBTEST.COM
I20260812 02:36:56.156795 16444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:56.132994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51326 (local address 127.7.132.130:36973)
0812 02:36:56.134690 (+ 1696us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:56.134697 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:56.134723 (+ 26us) server_negotiation.cc:407] Connection header received
0812 02:36:56.134783 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:56.134789 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:56.134991 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:56.135169 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:56.136748 (+ 1579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:56.137772 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:56.139922 (+ 2150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:56.140805 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:56.140917 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:56.148376 (+ 7459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:56.148393 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:56.148410 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:56.148473 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:56.152745 (+ 4272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:56.153732 (+ 987us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:56.153744 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:56.153748 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:56.153926 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:56.154540 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:56.154546 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:56.154549 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:56.155007 (+ 458us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:56.155319 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:56.156034 (+ 715us) server_negotiation.cc:299] Negotiation successful
0812 02:36:56.156314 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1545,"thread_start_us":1374,"threads_started":1}
I20260812 02:36:56.159310 15623 generic_service.cc:196] Checking for leaks (request via RPC)
Aug 12 02:36:56 dist-test-slave-mn4r krb5kdc[15163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502216, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.132.129@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM
I20260812 02:36:56.165546 16440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:56.127776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:50185 (local address 127.7.132.189:38815)
0812 02:36:56.131364 (+ 3588us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:56.131369 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:56.146488 (+ 15119us) server_negotiation.cc:407] Connection header received
0812 02:36:56.146809 (+ 321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:56.146818 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:56.146925 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:56.147042 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:56.149099 (+ 2057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:56.150567 (+ 1468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:56.152646 (+ 2079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:56.153203 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:56.153343 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:56.157650 (+ 4307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:56.157657 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:56.157661 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:56.157706 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:56.161338 (+ 3632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:56.162406 (+ 1068us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:56.162412 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:56.162415 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:56.162569 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:56.163098 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:56.163103 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:56.163106 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:56.163527 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:56.163662 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:56.164786 (+ 1124us) server_negotiation.cc:299] Negotiation successful
0812 02:36:56.165074 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3467,"thread_start_us":3312,"threads_started":1}
I20260812 02:36:56.165707 16275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:56.127473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:59679 (local address 127.7.132.188:39485)
0812 02:36:56.128187 (+ 714us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:56.128197 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:56.138953 (+ 10756us) server_negotiation.cc:407] Connection header received
0812 02:36:56.142041 (+ 3088us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:56.142049 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:56.142193 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:56.142276 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:56.144932 (+ 2656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:56.146053 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:56.148955 (+ 2902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:56.149758 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:56.149873 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:56.156156 (+ 6283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:56.156163 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:56.156168 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:56.156210 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:56.160665 (+ 4455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:56.162175 (+ 1510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:56.162184 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:56.162191 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:56.162395 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:56.163151 (+ 756us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:56.163169 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:56.163175 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:56.163640 (+ 465us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:56.163791 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:56.164841 (+ 1050us) server_negotiation.cc:299] Negotiation successful
0812 02:36:56.165190 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":517}
I20260812 02:36:56.168403 16442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:56.127815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.189:38815 (local address 127.7.132.129:50185)
0812 02:36:56.146342 (+ 18527us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:56.146367 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:56.146629 (+ 262us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:56.147465 (+ 836us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:56.147473 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:56.148332 (+ 859us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:36:56.148853 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:56.148870 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:56.150767 (+ 1897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:56.150774 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:56.152489 (+ 1715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:56.152501 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:56.153393 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:56.153399 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:56.153573 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:56.154287 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:36:56.154307 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:36:56.157419 (+ 3112us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:36:56.161536 (+ 4117us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:56.161541 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:56.161545 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:56.162264 (+ 719us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:56.162733 (+ 469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:56.162738 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:56.162741 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:56.162942 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:56.163770 (+ 828us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:36:56.163778 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:36:56.164327 (+ 549us) client_negotiation.cc:769] Sending connection context
0812 02:36:56.166166 (+ 1839us) client_negotiation.cc:240] Negotiation successful
0812 02:36:56.166674 (+ 508us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18290,"mutex_wait_us":62,"spinlock_wait_cycles":73472,"thread_start_us":17783,"threads_started":1}
I20260812 02:36:56.168437 16441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:56.127815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:39485 (local address 127.7.132.129:59679)
0812 02:36:56.138782 (+ 10967us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:56.138839 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:56.141673 (+ 2834us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:56.142874 (+ 1201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:56.142886 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:56.143740 (+ 854us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:36:56.144703 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:56.144734 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:56.146306 (+ 1572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:56.146312 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:56.148765 (+ 2453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:56.148786 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:56.150266 (+ 1480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:56.150274 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:56.150644 (+ 370us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:56.151440 (+ 796us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:36:56.151473 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:36:56.155879 (+ 4406us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:36:56.160924 (+ 5045us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:56.160936 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:56.160959 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:56.161940 (+ 981us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:56.162609 (+ 669us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:56.162617 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:56.162624 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:56.162984 (+ 360us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:56.163929 (+ 945us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:36:56.163937 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:36:56.164339 (+ 402us) client_negotiation.cc:769] Sending connection context
0812 02:36:56.166045 (+ 1706us) client_negotiation.cc:240] Negotiation successful
0812 02:36:56.167360 (+ 1315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10628,"mutex_wait_us":193,"spinlock_wait_cycles":9344,"thread_start_us":10357,"threads_started":1}
I20260812 02:36:56.170544 16437 heartbeater.cc:347] Connected to a master server at 127.7.132.189:38815
I20260812 02:36:56.170614 16435 heartbeater.cc:347] Connected to a master server at 127.7.132.188:39485
I20260812 02:36:56.170909 16435 heartbeater.cc:464] Registering TS with master...
I20260812 02:36:56.170912 16437 heartbeater.cc:464] Registering TS with master...
I20260812 02:36:56.171473 16445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:56.139973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:42365 (local address 127.7.132.190:33723)
0812 02:36:56.144086 (+ 4113us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:56.144092 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:56.150832 (+ 6740us) server_negotiation.cc:407] Connection header received
0812 02:36:56.151249 (+ 417us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:56.151256 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:56.151342 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:56.151421 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:56.154052 (+ 2631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:56.155258 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:56.157353 (+ 2095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:56.158052 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:56.158202 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:56.163170 (+ 4968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:56.163179 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:56.163187 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:56.163229 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:56.167309 (+ 4080us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:56.168162 (+ 853us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:56.168167 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:56.168170 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:56.168353 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:56.169429 (+ 1076us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:56.169435 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:56.169439 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:56.169825 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:56.169942 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:56.170577 (+ 635us) server_negotiation.cc:299] Negotiation successful
0812 02:36:56.170919 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3998,"thread_start_us":3886,"threads_started":1}
I20260812 02:36:56.172066 16437 heartbeater.cc:511] Master 127.7.132.189:38815 requested a full tablet report, sending...
I20260812 02:36:56.172106 16435 heartbeater.cc:511] Master 127.7.132.188:39485 requested a full tablet report, sending...
I20260812 02:36:56.172751 16446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:56.144094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:33723 (local address 127.7.132.129:42365)
0812 02:36:56.150730 (+ 6636us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:56.150749 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:56.151077 (+ 328us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:56.151654 (+ 577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:56.151658 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:56.152499 (+ 841us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:36:56.153168 (+ 669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:56.153183 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:56.155469 (+ 2286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:56.155475 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:56.157168 (+ 1693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:56.157185 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:56.158546 (+ 1361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:56.158556 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:56.159507 (+ 951us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:56.160085 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:36:56.160108 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:36:56.162950 (+ 2842us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:36:56.167498 (+ 4548us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:56.167502 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:56.167507 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:56.168016 (+ 509us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:56.169111 (+ 1095us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:56.169117 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:56.169120 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:56.169289 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:56.170100 (+ 811us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:36:56.170105 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:36:56.170273 (+ 168us) client_negotiation.cc:769] Sending connection context
0812 02:36:56.171921 (+ 1648us) client_negotiation.cc:240] Negotiation successful
0812 02:36:56.172327 (+ 406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6497,"thread_start_us":6759,"threads_started":1}
I20260812 02:36:56.173990 16436 heartbeater.cc:347] Connected to a master server at 127.7.132.190:33723
I20260812 02:36:56.174185 16436 heartbeater.cc:464] Registering TS with master...
I20260812 02:36:56.174724 16436 heartbeater.cc:511] Master 127.7.132.190:33723 requested a full tablet report, sending...
I20260812 02:36:56.174777 15984 ts_manager.cc:194] Registered new tserver with Master: ed2e8f19bfd642ac95e6b32d054ce8f4 (127.7.132.129:34169)
I20260812 02:36:56.175078 16215 ts_manager.cc:194] Registered new tserver with Master: ed2e8f19bfd642ac95e6b32d054ce8f4 (127.7.132.129:34169)
I20260812 02:36:56.176163 15223 ts_manager.cc:194] Re-registered known tserver with Master: ed2e8f19bfd642ac95e6b32d054ce8f4 (127.7.132.129:34169)
I20260812 02:36:56.177331 15223 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.132.129@KRBTEST.COM'} at 127.7.132.129:42365
I20260812 02:36:56.261189 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 15466
I20260812 02:36:56.283756 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.130:36973
--local_ip_for_outbound_sockets=127.7.132.130
--tserver_master_addrs=127.7.132.190:33723
--webserver_port=38257
--webserver_interface=127.7.132.130
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/krb5kdc/oryx_127.7.132.130.keytab
--principal=oryx/127.7.132.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:43909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.7.132.190:33723,127.7.132.189:38815
--tserver_master_addrs=127.7.132.190:33723,127.7.132.189:38815,127.7.132.188:39485 with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:36:56.624265 16449 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:36:56.624574 16449 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:36:56.624737 16449 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:36:56.632087 16449 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:36:56.632232 16449 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:36:56.632403 16449 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.130
I20260812 02:36:56.642408 16449 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:43909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/krb5kdc/oryx_127.7.132.130.keytab
--principal=oryx/127.7.132.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.130:36973
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.132.130
--webserver_port=38257
--tserver_master_addrs=127.7.132.190:33723,127.7.132.189:38815,127.7.132.188:39485
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.130
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:36:56.643743 16449 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:36:56.645188 16449 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:36:56.659190 16455 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:36:56.659394 16459 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:36:56.659493 16456 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:36:56.660790 16449 server_base.cc:1034] running on GCE node
I20260812 02:36:56.661414 16449 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:36:56.662421 16449 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:36:56.663646 16449 hybrid_clock.cc:648] HybridClock initialized: now 1786502216663595 us; error 45 us; skew 500 ppm
I20260812 02:36:56.663959 16449 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:36:56 dist-test-slave-mn4r krb5kdc[15163](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502216, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.132.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:36:56.667405 16449 init.cc:377] Logged in from keytab as oryx/127.7.132.130@KRBTEST.COM (short username oryx)
I20260812 02:36:56.668923 16449 webserver.cc:460] Webserver started at http://127.7.132.130:38257/ using document root <none> and password file <none>
I20260812 02:36:56.669612 16449 fs_manager.cc:359] Metadata directory not provided
I20260812 02:36:56.669757 16449 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:36:56.675315 16449 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260812 02:36:56.677968 16468 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:36:56.679330 16449 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 02:36:56.679497 16449 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/ts-1/wal
uuid: "07b5778ae04246ec9f152a523a1ef33f"
format_stamp: "Formatted at 2026-08-12 02:36:53 on dist-test-slave-mn4r"
server_key: "7077c4798e2ada1d7fdc890ba8c31ee4"
server_key_iv: "ba9000397d73642e69786e6ded1f6a82"
server_key_version: "encryptionkey@0"
I20260812 02:36:56.680481 16449 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:36:56.699765 16449 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:36:56.704026 16449 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:36:56.704324 16449 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:36:56.705317 16449 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:36:56.706727 16449 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:36:56.706843 16449 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:36:56.706988 16449 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:36:56.707083 16449 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:36:56.741616 16449 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.130:36973
I20260812 02:36:56.741724 16605 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.130:36973 every 8 connection(s)
I20260812 02:36:56.743180 16449 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/ts-1/data/info.pb
I20260812 02:36:56.748616 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 16449
I20260812 02:36:56.748951 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 15639
Aug 12 02:36:56 dist-test-slave-mn4r krb5kdc[15163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502216, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.132.130@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM
Aug 12 02:36:56 dist-test-slave-mn4r krb5kdc[15163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502216, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.132.130@KRBTEST.COM for oryx/127.7.132.189@KRBTEST.COM
Aug 12 02:36:56 dist-test-slave-mn4r krb5kdc[15163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502211, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.132.131@KRBTEST.COM
I20260812 02:36:56.785244 16619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:56.757630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:36853 (local address 127.7.132.190:33723)
0812 02:36:56.758760 (+ 1130us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:56.758766 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:56.764553 (+ 5787us) server_negotiation.cc:407] Connection header received
0812 02:36:56.764672 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:56.764679 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:56.764782 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:56.764961 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:56.766873 (+ 1912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:56.768100 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:56.770869 (+ 2769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:56.771600 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:56.771699 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:56.776724 (+ 5025us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:56.776732 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:56.776737 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:56.776783 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:56.780528 (+ 3745us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:56.781666 (+ 1138us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:56.781675 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:56.781681 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:56.781858 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:56.782509 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:56.782514 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:56.782518 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:56.782957 (+ 439us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:56.783116 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:56.784265 (+ 1149us) server_negotiation.cc:299] Negotiation successful
0812 02:36:56.784671 (+ 406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":999,"thread_start_us":877,"threads_started":1}
I20260812 02:36:56.785423 16612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:56.746536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:58043 (local address 127.7.132.189:38815)
0812 02:36:56.758933 (+ 12397us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:56.758941 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:56.759699 (+ 758us) server_negotiation.cc:407] Connection header received
0812 02:36:56.763623 (+ 3924us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:56.763630 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:56.763736 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:56.763872 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:56.766917 (+ 3045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:56.768410 (+ 1493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:56.771111 (+ 2701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:56.771961 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:56.772085 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:56.777428 (+ 5343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:56.777438 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:56.777445 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:56.777501 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:56.781360 (+ 3859us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:56.782407 (+ 1047us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:56.782413 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:56.782417 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:56.782562 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:56.783097 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:56.783103 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:56.783106 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:56.783507 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:56.783660 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:56.784676 (+ 1016us) server_negotiation.cc:299] Negotiation successful
0812 02:36:56.785026 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12238,"thread_start_us":12418,"threads_started":1}
Aug 12 02:36:56 dist-test-slave-mn4r krb5kdc[15163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502216, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.132.130@KRBTEST.COM for oryx/127.7.132.188@KRBTEST.COM
I20260812 02:36:56.787920 16616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:56.747053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.189:38815 (local address 127.7.132.130:58043)
0812 02:36:56.759477 (+ 12424us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:56.759559 (+ 82us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:56.763169 (+ 3610us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:56.764444 (+ 1275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:56.764461 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:56.765488 (+ 1027us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:36:56.766656 (+ 1168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:56.766684 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:56.768661 (+ 1977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:56.768669 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:56.770889 (+ 2220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:56.770911 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:56.772480 (+ 1569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:56.772489 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:56.772749 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:56.773360 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:36:56.773393 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:36:56.777118 (+ 3725us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:36:56.781525 (+ 4407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:56.781530 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:56.781534 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:56.782158 (+ 624us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:56.782727 (+ 569us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:56.782732 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:56.782735 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:56.782916 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:56.783745 (+ 829us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:36:56.783749 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:36:56.784011 (+ 262us) client_negotiation.cc:769] Sending connection context
0812 02:36:56.785616 (+ 1605us) client_negotiation.cc:240] Negotiation successful
0812 02:36:56.786444 (+ 828us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12032,"mutex_wait_us":285,"thread_start_us":16948,"threads_started":1}
I20260812 02:36:56.787945 16614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:56.758763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:33723 (local address 127.7.132.130:36853)
0812 02:36:56.764385 (+ 5622us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:56.764404 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:56.764553 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:56.765151 (+ 598us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:56.765157 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:56.765886 (+ 729us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:36:56.766652 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:56.766680 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:56.768303 (+ 1623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:56.768308 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:56.770699 (+ 2391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:56.770714 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:56.772038 (+ 1324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:56.772046 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:56.772320 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:56.773193 (+ 873us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:36:56.773230 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:36:56.776430 (+ 3200us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:36:56.780711 (+ 4281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:56.780723 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:56.780743 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:56.781443 (+ 700us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:56.782042 (+ 599us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:56.782047 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:56.782050 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:56.782355 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:56.783222 (+ 867us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:36:56.783233 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:36:56.783722 (+ 489us) client_negotiation.cc:769] Sending connection context
0812 02:36:56.785574 (+ 1852us) client_negotiation.cc:240] Negotiation successful
0812 02:36:56.786318 (+ 744us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5499,"mutex_wait_us":38,"thread_start_us":5000,"threads_started":1}
I20260812 02:36:56.789675 16609 heartbeater.cc:347] Connected to a master server at 127.7.132.190:33723
I20260812 02:36:56.790022 16609 heartbeater.cc:464] Registering TS with master...
I20260812 02:36:56.790800 16609 heartbeater.cc:511] Master 127.7.132.190:33723 requested a full tablet report, sending...
I20260812 02:36:56.789723 16610 heartbeater.cc:347] Connected to a master server at 127.7.132.189:38815
I20260812 02:36:56.791642 16610 heartbeater.cc:464] Registering TS with master...
I20260812 02:36:56.792168 16610 heartbeater.cc:511] Master 127.7.132.189:38815 requested a full tablet report, sending...
I20260812 02:36:56.792464 15223 ts_manager.cc:194] Re-registered known tserver with Master: 07b5778ae04246ec9f152a523a1ef33f (127.7.132.130:36973)
I20260812 02:36:56.793406 15984 ts_manager.cc:194] Registered new tserver with Master: 07b5778ae04246ec9f152a523a1ef33f (127.7.132.130:36973)
I20260812 02:36:56.793646 15223 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.132.130@KRBTEST.COM'} at 127.7.132.130:36853
I20260812 02:36:56.794749 16620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:56.747053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:39485 (local address 127.7.132.130:58161)
0812 02:36:56.760890 (+ 13837us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:56.760908 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:56.763689 (+ 2781us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:56.778502 (+ 14813us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:56.778508 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:56.779124 (+ 616us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:36:56.779465 (+ 341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:56.779485 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:56.781049 (+ 1564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:56.781056 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:56.782753 (+ 1697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:56.782767 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:56.783834 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:56.783841 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:56.784053 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:56.784641 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:36:56.784662 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:36:56.787444 (+ 2782us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:36:56.791766 (+ 4322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:56.791772 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:56.791776 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:56.792185 (+ 409us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:56.792682 (+ 497us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:56.792687 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:56.792690 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:56.792883 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:56.793586 (+ 703us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:36:56.793592 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:36:56.793753 (+ 161us) client_negotiation.cc:769] Sending connection context
0812 02:36:56.793886 (+ 133us) client_negotiation.cc:240] Negotiation successful
0812 02:36:56.794263 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13673,"mutex_wait_us":107,"spinlock_wait_cycles":72576,"thread_start_us":8675,"threads_started":1}
I20260812 02:36:56.795626 16613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:56.746518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:58161 (local address 127.7.132.188:39485)
0812 02:36:56.778138 (+ 31620us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:56.778145 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:56.778166 (+ 21us) server_negotiation.cc:407] Connection header received
0812 02:36:56.778230 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:56.778235 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:56.778330 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:56.778452 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:56.779676 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:56.780868 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:56.782945 (+ 2077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:56.783604 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:56.784361 (+ 757us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:56.787698 (+ 3337us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:56.787705 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:56.787710 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:56.787752 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:56.791531 (+ 3779us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:56.792362 (+ 831us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:56.792374 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:56.792378 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:56.792540 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:56.793044 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:56.793048 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:56.793051 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:56.793433 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:56.794868 (+ 1435us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:56.795027 (+ 159us) server_negotiation.cc:299] Negotiation successful
0812 02:36:56.795267 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31436,"thread_start_us":31309,"threads_started":1}
I20260812 02:36:56.796315 16607 heartbeater.cc:347] Connected to a master server at 127.7.132.188:39485
I20260812 02:36:56.796461 16607 heartbeater.cc:464] Registering TS with master...
I20260812 02:36:56.796887 16607 heartbeater.cc:511] Master 127.7.132.188:39485 requested a full tablet report, sending...
I20260812 02:36:56.797756 16618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:56.750587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37464 (local address 127.7.132.131:35367)
0812 02:36:56.765200 (+ 14613us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:56.765212 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:56.766124 (+ 912us) server_negotiation.cc:407] Connection header received
0812 02:36:56.766206 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:56.766214 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:56.766485 (+ 271us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:56.766664 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:56.769473 (+ 2809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:56.771184 (+ 1711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:56.773628 (+ 2444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:56.774564 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:56.774706 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:56.783966 (+ 9260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:56.783983 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:56.784000 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:56.784102 (+ 102us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:56.792565 (+ 8463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:56.793590 (+ 1025us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:56.793600 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:56.793604 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:56.793786 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:56.794784 (+ 998us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:56.794792 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:56.794797 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:56.795206 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:56.795539 (+ 333us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:56.795889 (+ 350us) server_negotiation.cc:299] Negotiation successful
0812 02:36:56.797448 (+ 1559us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14427,"thread_start_us":14204,"threads_started":1}
I20260812 02:36:56.798065 16215 ts_manager.cc:194] Registered new tserver with Master: 07b5778ae04246ec9f152a523a1ef33f (127.7.132.130:36973)
I20260812 02:36:56.799077 15797 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:36:56.889412 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 15639
I20260812 02:36:56.911873 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.131:35367
--local_ip_for_outbound_sockets=127.7.132.131
--tserver_master_addrs=127.7.132.190:33723
--webserver_port=45735
--webserver_interface=127.7.132.131
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/krb5kdc/oryx_127.7.132.131.keytab
--principal=oryx/127.7.132.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:43909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.7.132.190:33723,127.7.132.189:38815
--tserver_master_addrs=127.7.132.190:33723,127.7.132.189:38815,127.7.132.188:39485 with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 02:36:57.180286 16436 heartbeater.cc:503] Master 127.7.132.190:33723 was elected leader, sending a full tablet report...
W20260812 02:36:57.268152 16622 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:36:57.268482 16622 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:36:57.268656 16622 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:36:57.276261 16622 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:36:57.276403 16622 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:36:57.276587 16622 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.131
I20260812 02:36:57.286751 16622 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:43909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/krb5kdc/oryx_127.7.132.131.keytab
--principal=oryx/127.7.132.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.131:35367
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.132.131
--webserver_port=45735
--tserver_master_addrs=127.7.132.190:33723,127.7.132.189:38815,127.7.132.188:39485
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.131
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:36:57.288184 16622 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:36:57.289701 16622 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:36:57.303576 16630 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:36:57.303678 16631 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:36:57.303900 16634 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:36:57.304816 16622 server_base.cc:1034] running on GCE node
I20260812 02:36:57.305437 16622 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:36:57.306488 16622 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:36:57.307704 16622 hybrid_clock.cc:648] HybridClock initialized: now 1786502217307656 us; error 58 us; skew 500 ppm
I20260812 02:36:57.308032 16622 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:36:57 dist-test-slave-mn4r krb5kdc[15163](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502217, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.132.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:36:57.311285 16622 init.cc:377] Logged in from keytab as oryx/127.7.132.131@KRBTEST.COM (short username oryx)
I20260812 02:36:57.312844 16622 webserver.cc:460] Webserver started at http://127.7.132.131:45735/ using document root <none> and password file <none>
I20260812 02:36:57.313565 16622 fs_manager.cc:359] Metadata directory not provided
I20260812 02:36:57.313743 16622 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:36:57.319478 16622 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 02:36:57.322597 16641 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:36:57.324083 16622 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260812 02:36:57.324259 16622 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/ts-2/wal
uuid: "6c404a3267bc4a11a6850d91dec7b21d"
format_stamp: "Formatted at 2026-08-12 02:36:53 on dist-test-slave-mn4r"
server_key: "7de24234a859286210c3aca800c86795"
server_key_iv: "605bf975428e615b167ec11ea392d64c"
server_key_version: "encryptionkey@0"
I20260812 02:36:57.325366 16622 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:36:57.357764 16622 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:36:57.362318 16622 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:36:57.362623 16622 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:36:57.363692 16622 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:36:57.365137 16622 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:36:57.365226 16622 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:36:57.365378 16622 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:36:57.365490 16622 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:36:57.400162 16622 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.131:35367
I20260812 02:36:57.400252 16778 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.131:35367 every 8 connection(s)
I20260812 02:36:57.401645 16622 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/ts-2/data/info.pb
I20260812 02:36:57.408140 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 16622
Aug 12 02:36:57 dist-test-slave-mn4r krb5kdc[15163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502217, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.132.131@KRBTEST.COM for oryx/127.7.132.189@KRBTEST.COM
Aug 12 02:36:57 dist-test-slave-mn4r krb5kdc[15163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502217, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.132.131@KRBTEST.COM for oryx/127.7.132.188@KRBTEST.COM
Aug 12 02:36:57 dist-test-slave-mn4r krb5kdc[15163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502217, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.132.131@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM
I20260812 02:36:57.449805 16787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:57.405477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.189:38815 (local address 127.7.132.131:43215)
0812 02:36:57.406711 (+ 1234us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:57.406772 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:57.410283 (+ 3511us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:57.415791 (+ 5508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:57.415805 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:57.416712 (+ 907us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:36:57.417669 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:57.417699 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:57.419555 (+ 1856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:57.419560 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:57.422376 (+ 2816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:57.422399 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:57.423902 (+ 1503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:57.423913 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:57.424304 (+ 391us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:57.425130 (+ 826us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:36:57.425181 (+ 51us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:36:57.428295 (+ 3114us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:36:57.443354 (+ 15059us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:57.443365 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:57.443399 (+ 34us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:57.444142 (+ 743us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:57.445124 (+ 982us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:57.445131 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:57.445136 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:57.445460 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:57.446444 (+ 984us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:36:57.446458 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:36:57.447017 (+ 559us) client_negotiation.cc:769] Sending connection context
0812 02:36:57.447380 (+ 363us) client_negotiation.cc:240] Negotiation successful
0812 02:36:57.448081 (+ 701us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":817,"thread_start_us":566,"threads_started":1}
I20260812 02:36:57.451682 16785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:57.404887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:43215 (local address 127.7.132.189:38815)
0812 02:36:57.415066 (+ 10179us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:57.415076 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:57.415101 (+ 25us) server_negotiation.cc:407] Connection header received
0812 02:36:57.415187 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:57.415192 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:57.415304 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:57.415453 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:57.417919 (+ 2466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:57.419142 (+ 1223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:57.422753 (+ 3611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:57.423558 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:57.423732 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:57.428557 (+ 4825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:57.428563 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:57.428567 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:57.428605 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:57.441550 (+ 12945us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:57.444405 (+ 2855us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:57.444412 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:57.444416 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:57.444591 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:57.445675 (+ 1084us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:57.445682 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:57.445686 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:57.446124 (+ 438us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:57.450776 (+ 4652us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:57.450944 (+ 168us) server_negotiation.cc:299] Negotiation successful
0812 02:36:57.451201 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10028,"thread_start_us":10023,"threads_started":1}
I20260812 02:36:57.453272 16781 heartbeater.cc:347] Connected to a master server at 127.7.132.189:38815
I20260812 02:36:57.453694 16781 heartbeater.cc:464] Registering TS with master...
I20260812 02:36:57.454797 16781 heartbeater.cc:511] Master 127.7.132.189:38815 requested a full tablet report, sending...
I20260812 02:36:57.455621 16784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:57.404797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:34633 (local address 127.7.132.188:39485)
0812 02:36:57.414568 (+ 9771us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:57.414577 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:57.416182 (+ 1605us) server_negotiation.cc:407] Connection header received
0812 02:36:57.416460 (+ 278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:57.416466 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:57.416558 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:57.416646 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:57.417963 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:57.419150 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:57.425204 (+ 6054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:57.426002 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:57.426222 (+ 220us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:57.442406 (+ 16184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:57.442418 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:57.442426 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:57.442512 (+ 86us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:57.451472 (+ 8960us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:57.452428 (+ 956us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:57.452434 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:57.452438 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:57.452599 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:57.453024 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:57.453041 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:57.453044 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:57.453407 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:57.453531 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:57.454963 (+ 1432us) server_negotiation.cc:299] Negotiation successful
0812 02:36:57.455233 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9627,"thread_start_us":9513,"threads_started":1}
I20260812 02:36:57.456954 15984 ts_manager.cc:194] Registered new tserver with Master: 6c404a3267bc4a11a6850d91dec7b21d (127.7.132.131:35367)
I20260812 02:36:57.458114 16788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:57.405477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:39485 (local address 127.7.132.131:34633)
0812 02:36:57.416048 (+ 10571us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:57.416073 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:57.416297 (+ 224us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:57.416881 (+ 584us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:57.416886 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:57.417496 (+ 610us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:36:57.417788 (+ 292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:57.417801 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:57.419376 (+ 1575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:57.419382 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:57.424991 (+ 5609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:57.425011 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:57.431678 (+ 6667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:57.431688 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:57.432449 (+ 761us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:57.433276 (+ 827us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:36:57.433307 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:36:57.442134 (+ 8827us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:36:57.451623 (+ 9489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:57.451628 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:57.451633 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:57.452212 (+ 579us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:57.452741 (+ 529us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:57.452746 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:57.452750 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:57.452892 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:57.453607 (+ 715us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:36:57.453612 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:36:57.453786 (+ 174us) client_negotiation.cc:769] Sending connection context
0812 02:36:57.453921 (+ 135us) client_negotiation.cc:240] Negotiation successful
0812 02:36:57.454288 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10396,"mutex_wait_us":432,"thread_start_us":9768,"threads_started":1}
I20260812 02:36:57.459077 16779 heartbeater.cc:347] Connected to a master server at 127.7.132.188:39485
I20260812 02:36:57.459228 16779 heartbeater.cc:464] Registering TS with master...
I20260812 02:36:57.459820 16779 heartbeater.cc:511] Master 127.7.132.188:39485 requested a full tablet report, sending...
I20260812 02:36:57.459949 16794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:57.419930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:33723 (local address 127.7.132.131:42803)
0812 02:36:57.423635 (+ 3705us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:57.423656 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:57.423796 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:57.430571 (+ 6775us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:57.430580 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:57.431346 (+ 766us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:36:57.431879 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:57.431908 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:57.434429 (+ 2521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:57.434438 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:57.436862 (+ 2424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:57.436881 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:57.438153 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:57.438164 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:57.438427 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:57.439166 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:36:57.439199 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:36:57.443264 (+ 4065us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:36:57.455842 (+ 12578us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:57.455849 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:57.455852 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:57.456413 (+ 561us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:57.457205 (+ 792us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:36:57.457212 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:36:57.457218 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:36:57.457385 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:36:57.458805 (+ 1420us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:36:57.458812 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:36:57.458966 (+ 154us) client_negotiation.cc:769] Sending connection context
0812 02:36:57.459128 (+ 162us) client_negotiation.cc:240] Negotiation successful
0812 02:36:57.459485 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3526,"thread_start_us":3458,"threads_started":1}
I20260812 02:36:57.461182 16215 ts_manager.cc:194] Registered new tserver with Master: 6c404a3267bc4a11a6850d91dec7b21d (127.7.132.131:35367)
I20260812 02:36:57.461414 16792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:57.416562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:42803 (local address 127.7.132.190:33723)
0812 02:36:57.423353 (+ 6791us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:57.423360 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:57.424701 (+ 1341us) server_negotiation.cc:407] Connection header received
0812 02:36:57.424790 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:57.424797 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:57.426879 (+ 2082us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:57.427032 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:57.432151 (+ 5119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:57.434198 (+ 2047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:57.437048 (+ 2850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:57.437919 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:57.438067 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:57.443945 (+ 5878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:57.443954 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:57.443960 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:57.444007 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:57.455238 (+ 11231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:57.456734 (+ 1496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:57.456741 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:57.456746 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:57.456991 (+ 245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:57.457651 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:57.457656 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:57.457659 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:57.458071 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:57.458559 (+ 488us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:57.460666 (+ 2107us) server_negotiation.cc:299] Negotiation successful
0812 02:36:57.460936 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6642,"thread_start_us":6523,"threads_started":1}
I20260812 02:36:57.462498 16780 heartbeater.cc:347] Connected to a master server at 127.7.132.190:33723
I20260812 02:36:57.462646 16780 heartbeater.cc:464] Registering TS with master...
I20260812 02:36:57.463100 16780 heartbeater.cc:511] Master 127.7.132.190:33723 requested a full tablet report, sending...
I20260812 02:36:57.464134 15223 ts_manager.cc:194] Re-registered known tserver with Master: 6c404a3267bc4a11a6850d91dec7b21d (127.7.132.131:35367)
I20260812 02:36:57.465407 15223 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.132.131@KRBTEST.COM'} at 127.7.132.131:42803
I20260812 02:36:57.466291 16797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:57.430820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41040 (local address 127.7.132.189:38815)
0812 02:36:57.446368 (+ 15548us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:57.446377 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:57.446401 (+ 24us) server_negotiation.cc:407] Connection header received
0812 02:36:57.446465 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:57.446473 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:57.447167 (+ 694us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:57.447298 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:57.449790 (+ 2492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:57.451017 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:57.453383 (+ 2366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:57.454033 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:57.454868 (+ 835us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:57.458543 (+ 3675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:57.458551 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:57.458555 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:57.458593 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:57.462217 (+ 3624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:57.463106 (+ 889us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:57.463112 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:57.463115 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:57.463274 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:57.463763 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:57.463768 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:57.463772 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:57.464099 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:57.465532 (+ 1433us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:57.465649 (+ 117us) server_negotiation.cc:299] Negotiation successful
0812 02:36:57.465916 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15364,"thread_start_us":15398,"threads_started":1}
I20260812 02:36:57.474864 16800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:57.435364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35110 (local address 127.7.132.188:39485)
0812 02:36:57.436086 (+ 722us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:57.436095 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:57.449658 (+ 13563us) server_negotiation.cc:407] Connection header received
0812 02:36:57.450391 (+ 733us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:57.450397 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:57.450486 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:57.450569 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:57.459780 (+ 9211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:57.461100 (+ 1320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:57.463442 (+ 2342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:57.464124 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:57.464237 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:57.468799 (+ 4562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:57.468807 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:57.468811 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:57.468850 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:57.471700 (+ 2850us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:57.472602 (+ 902us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:57.472606 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:57.472608 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:57.472780 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:57.473231 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:57.473236 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:57.473239 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:57.473536 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:57.473789 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:57.474262 (+ 473us) server_negotiation.cc:299] Negotiation successful
0812 02:36:57.474441 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":508,"thread_start_us":327,"threads_started":1}
I20260812 02:36:57.475703 16795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:57.425506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33688 (local address 127.7.132.190:33723)
0812 02:36:57.448588 (+ 23082us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:57.448595 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:57.448615 (+ 20us) server_negotiation.cc:407] Connection header received
0812 02:36:57.448681 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:57.448687 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:57.448761 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:57.448875 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:57.456689 (+ 7814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:57.462036 (+ 5347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:57.464309 (+ 2273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:57.465016 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:57.465163 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:57.468825 (+ 3662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:57.468830 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:57.468833 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:57.468865 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:57.472592 (+ 3727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:57.473375 (+ 783us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:57.473379 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:57.473381 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:57.473529 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:57.473968 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:57.473974 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:57.473977 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:57.474411 (+ 434us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:57.474658 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:57.475107 (+ 449us) server_negotiation.cc:299] Negotiation successful
0812 02:36:57.475357 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22940,"thread_start_us":24173,"threads_started":1}
I20260812 02:36:57.479589 15223 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33688:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 02:36:57.487593 15223 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 02:36:57.519579 16821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:57.507595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.131:35367 (local address 127.0.0.1:37480)
0812 02:36:57.508780 (+ 1185us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:57.508808 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:57.508924 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:57.511059 (+ 2135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:57.511067 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:57.511101 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:36:57.511536 (+ 435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:57.511554 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:57.514389 (+ 2835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:57.514398 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:57.517234 (+ 2836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:57.517249 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:57.518741 (+ 1492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:57.518748 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:57.518962 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:57.518997 (+ 35us) client_negotiation.cc:769] Sending connection context
0812 02:36:57.519109 (+ 112us) client_negotiation.cc:240] Negotiation successful
0812 02:36:57.519259 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":981,"mutex_wait_us":113,"thread_start_us":3754,"threads_started":1}
I20260812 02:36:57.519709 16822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:57.508084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37480 (local address 127.7.132.131:35367)
0812 02:36:57.510540 (+ 2456us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:57.510548 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:57.510576 (+ 28us) server_negotiation.cc:407] Connection header received
0812 02:36:57.510663 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:57.510669 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:57.510871 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:57.511041 (+ 170us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:36:57.511932 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:57.514205 (+ 2273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:57.517392 (+ 3187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:57.518555 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:57.518659 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:57.518817 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:57.519198 (+ 381us) server_negotiation.cc:299] Negotiation successful
0812 02:36:57.519371 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2315,"thread_start_us":2155,"threads_started":1}
I20260812 02:36:57.521766 16819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:57.507183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55858 (local address 127.7.132.129:34169)
0812 02:36:57.514733 (+ 7550us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:57.514745 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:57.514773 (+ 28us) server_negotiation.cc:407] Connection header received
0812 02:36:57.514873 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:57.514883 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:57.515117 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:57.515314 (+ 197us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:36:57.515999 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:57.517060 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:57.519177 (+ 2117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:57.520628 (+ 1451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:57.520766 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:57.520959 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:57.521282 (+ 323us) server_negotiation.cc:299] Negotiation successful
0812 02:36:57.521462 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7378,"thread_start_us":7251,"threads_started":1}
I20260812 02:36:57.521808 16818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:57.506921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:34169 (local address 127.0.0.1:55858)
0812 02:36:57.507637 (+ 716us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:57.507659 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:57.507799 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:57.515306 (+ 7507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:57.515310 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:57.515341 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:36:57.515823 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:57.515832 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:57.517241 (+ 1409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:57.517247 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:57.519036 (+ 1789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:57.519050 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:57.520862 (+ 1812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:57.520871 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:57.521102 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:57.521134 (+ 32us) client_negotiation.cc:769] Sending connection context
0812 02:36:57.521235 (+ 101us) client_negotiation.cc:240] Negotiation successful
0812 02:36:57.521356 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":587,"mutex_wait_us":64,"thread_start_us":370,"threads_started":1}
I20260812 02:36:57.523840 16820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:57.508080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.130:36973 (local address 127.0.0.1:58164)
0812 02:36:57.511575 (+ 3495us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:57.511594 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:57.511743 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:57.512402 (+ 659us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:57.512409 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:57.512446 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:36:57.512935 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:57.512950 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:57.514957 (+ 2007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:57.514962 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:57.519182 (+ 4220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:57.519190 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:57.523025 (+ 3835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:57.523032 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:57.523202 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:57.523236 (+ 34us) client_negotiation.cc:769] Sending connection context
0812 02:36:57.523342 (+ 106us) client_negotiation.cc:240] Negotiation successful
0812 02:36:57.523468 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3398,"mutex_wait_us":47,"thread_start_us":289,"threads_started":1}
I20260812 02:36:57.524307 16700 tablet_service.cc:1467] Processing CreateTablet for tablet 362a5f2576704a9295784c42fdd3e7ea (DEFAULT_TABLE table=test-table [id=da33c73d3e09419eb75e4422507f1d00]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:36:57.525810 16700 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 362a5f2576704a9295784c42fdd3e7ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:36:57.525672 16353 tablet_service.cc:1467] Processing CreateTablet for tablet 362a5f2576704a9295784c42fdd3e7ea (DEFAULT_TABLE table=test-table [id=da33c73d3e09419eb75e4422507f1d00]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:36:57.526916 16353 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 362a5f2576704a9295784c42fdd3e7ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:36:57.529762 16824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:57.508391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58164 (local address 127.7.132.130:36973)
0812 02:36:57.508991 (+ 600us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:57.509000 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:57.511711 (+ 2711us) server_negotiation.cc:407] Connection header received
0812 02:36:57.511892 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:57.511901 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:57.512178 (+ 277us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:57.512398 (+ 220us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:36:57.513302 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:57.514805 (+ 1503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:57.521029 (+ 6224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:57.522819 (+ 1790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:57.522947 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:57.523141 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:57.529306 (+ 6165us) server_negotiation.cc:299] Negotiation successful
0812 02:36:57.529495 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":427,"thread_start_us":292,"threads_started":1}
I20260812 02:36:57.532577 16530 tablet_service.cc:1467] Processing CreateTablet for tablet 362a5f2576704a9295784c42fdd3e7ea (DEFAULT_TABLE table=test-table [id=da33c73d3e09419eb75e4422507f1d00]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:36:57.533748 16530 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 362a5f2576704a9295784c42fdd3e7ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:36:57.538872 16825 tablet_bootstrap.cc:492] T 362a5f2576704a9295784c42fdd3e7ea P ed2e8f19bfd642ac95e6b32d054ce8f4: Bootstrap starting.
I20260812 02:36:57.542057 16825 tablet_bootstrap.cc:654] T 362a5f2576704a9295784c42fdd3e7ea P ed2e8f19bfd642ac95e6b32d054ce8f4: Neither blocks nor log segments found. Creating new log.
I20260812 02:36:57.543467 16825 log.cc:824] T 362a5f2576704a9295784c42fdd3e7ea P ed2e8f19bfd642ac95e6b32d054ce8f4: Log is configured to *not* fsync() on all Append() calls
I20260812 02:36:57.544940 16826 tablet_bootstrap.cc:492] T 362a5f2576704a9295784c42fdd3e7ea P 6c404a3267bc4a11a6850d91dec7b21d: Bootstrap starting.
I20260812 02:36:57.546757 16825 tablet_bootstrap.cc:492] T 362a5f2576704a9295784c42fdd3e7ea P ed2e8f19bfd642ac95e6b32d054ce8f4: No bootstrap required, opened a new log
I20260812 02:36:57.547034 16825 ts_tablet_manager.cc:1397] T 362a5f2576704a9295784c42fdd3e7ea P ed2e8f19bfd642ac95e6b32d054ce8f4: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.005s
I20260812 02:36:57.548921 16826 tablet_bootstrap.cc:654] T 362a5f2576704a9295784c42fdd3e7ea P 6c404a3267bc4a11a6850d91dec7b21d: Neither blocks nor log segments found. Creating new log.
I20260812 02:36:57.549408 16828 tablet_bootstrap.cc:492] T 362a5f2576704a9295784c42fdd3e7ea P 07b5778ae04246ec9f152a523a1ef33f: Bootstrap starting.
I20260812 02:36:57.550186 16826 log.cc:824] T 362a5f2576704a9295784c42fdd3e7ea P 6c404a3267bc4a11a6850d91dec7b21d: Log is configured to *not* fsync() on all Append() calls
I20260812 02:36:57.552703 16828 tablet_bootstrap.cc:654] T 362a5f2576704a9295784c42fdd3e7ea P 07b5778ae04246ec9f152a523a1ef33f: Neither blocks nor log segments found. Creating new log.
I20260812 02:36:57.553471 16826 tablet_bootstrap.cc:492] T 362a5f2576704a9295784c42fdd3e7ea P 6c404a3267bc4a11a6850d91dec7b21d: No bootstrap required, opened a new log
I20260812 02:36:57.553655 16828 log.cc:824] T 362a5f2576704a9295784c42fdd3e7ea P 07b5778ae04246ec9f152a523a1ef33f: Log is configured to *not* fsync() on all Append() calls
I20260812 02:36:57.553730 16826 ts_tablet_manager.cc:1397] T 362a5f2576704a9295784c42fdd3e7ea P 6c404a3267bc4a11a6850d91dec7b21d: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260812 02:36:57.556520 16828 tablet_bootstrap.cc:492] T 362a5f2576704a9295784c42fdd3e7ea P 07b5778ae04246ec9f152a523a1ef33f: No bootstrap required, opened a new log
I20260812 02:36:57.556784 16828 ts_tablet_manager.cc:1397] T 362a5f2576704a9295784c42fdd3e7ea P 07b5778ae04246ec9f152a523a1ef33f: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260812 02:36:57.557670 16825 raft_consensus.cc:348] T 362a5f2576704a9295784c42fdd3e7ea P ed2e8f19bfd642ac95e6b32d054ce8f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed2e8f19bfd642ac95e6b32d054ce8f4" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 34169 } } peers { permanent_uuid: "6c404a3267bc4a11a6850d91dec7b21d" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 35367 } } peers { permanent_uuid: "07b5778ae04246ec9f152a523a1ef33f" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 36973 } }
I20260812 02:36:57.558149 16825 raft_consensus.cc:374] T 362a5f2576704a9295784c42fdd3e7ea P ed2e8f19bfd642ac95e6b32d054ce8f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:36:57.558259 16825 raft_consensus.cc:729] T 362a5f2576704a9295784c42fdd3e7ea P ed2e8f19bfd642ac95e6b32d054ce8f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed2e8f19bfd642ac95e6b32d054ce8f4, State: Initialized, Role: FOLLOWER
I20260812 02:36:57.558888 16825 consensus_queue.cc:260] T 362a5f2576704a9295784c42fdd3e7ea P ed2e8f19bfd642ac95e6b32d054ce8f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed2e8f19bfd642ac95e6b32d054ce8f4" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 34169 } } peers { permanent_uuid: "6c404a3267bc4a11a6850d91dec7b21d" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 35367 } } peers { permanent_uuid: "07b5778ae04246ec9f152a523a1ef33f" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 36973 } }
I20260812 02:36:57.563477 16825 ts_tablet_manager.cc:1428] T 362a5f2576704a9295784c42fdd3e7ea P ed2e8f19bfd642ac95e6b32d054ce8f4: Time spent starting tablet: real 0.016s user 0.016s sys 0.000s
I20260812 02:36:57.565053 16826 raft_consensus.cc:348] T 362a5f2576704a9295784c42fdd3e7ea P 6c404a3267bc4a11a6850d91dec7b21d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed2e8f19bfd642ac95e6b32d054ce8f4" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 34169 } } peers { permanent_uuid: "6c404a3267bc4a11a6850d91dec7b21d" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 35367 } } peers { permanent_uuid: "07b5778ae04246ec9f152a523a1ef33f" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 36973 } }
I20260812 02:36:57.565466 16826 raft_consensus.cc:374] T 362a5f2576704a9295784c42fdd3e7ea P 6c404a3267bc4a11a6850d91dec7b21d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:36:57.565307 16828 raft_consensus.cc:348] T 362a5f2576704a9295784c42fdd3e7ea P 07b5778ae04246ec9f152a523a1ef33f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed2e8f19bfd642ac95e6b32d054ce8f4" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 34169 } } peers { permanent_uuid: "6c404a3267bc4a11a6850d91dec7b21d" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 35367 } } peers { permanent_uuid: "07b5778ae04246ec9f152a523a1ef33f" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 36973 } }
I20260812 02:36:57.565570 16826 raft_consensus.cc:729] T 362a5f2576704a9295784c42fdd3e7ea P 6c404a3267bc4a11a6850d91dec7b21d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c404a3267bc4a11a6850d91dec7b21d, State: Initialized, Role: FOLLOWER
I20260812 02:36:57.565701 16828 raft_consensus.cc:374] T 362a5f2576704a9295784c42fdd3e7ea P 07b5778ae04246ec9f152a523a1ef33f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:36:57.565804 16828 raft_consensus.cc:729] T 362a5f2576704a9295784c42fdd3e7ea P 07b5778ae04246ec9f152a523a1ef33f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07b5778ae04246ec9f152a523a1ef33f, State: Initialized, Role: FOLLOWER
I20260812 02:36:57.566437 16828 consensus_queue.cc:260] T 362a5f2576704a9295784c42fdd3e7ea P 07b5778ae04246ec9f152a523a1ef33f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed2e8f19bfd642ac95e6b32d054ce8f4" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 34169 } } peers { permanent_uuid: "6c404a3267bc4a11a6850d91dec7b21d" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 35367 } } peers { permanent_uuid: "07b5778ae04246ec9f152a523a1ef33f" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 36973 } }
I20260812 02:36:57.570122 16826 consensus_queue.cc:260] T 362a5f2576704a9295784c42fdd3e7ea P 6c404a3267bc4a11a6850d91dec7b21d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed2e8f19bfd642ac95e6b32d054ce8f4" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 34169 } } peers { permanent_uuid: "6c404a3267bc4a11a6850d91dec7b21d" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 35367 } } peers { permanent_uuid: "07b5778ae04246ec9f152a523a1ef33f" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 36973 } }
I20260812 02:36:57.574383 16780 heartbeater.cc:503] Master 127.7.132.190:33723 was elected leader, sending a full tablet report...
I20260812 02:36:57.574605 16828 ts_tablet_manager.cc:1428] T 362a5f2576704a9295784c42fdd3e7ea P 07b5778ae04246ec9f152a523a1ef33f: Time spent starting tablet: real 0.018s user 0.010s sys 0.009s
I20260812 02:36:57.575026 16609 heartbeater.cc:503] Master 127.7.132.190:33723 was elected leader, sending a full tablet report...
I20260812 02:36:57.576517 16826 ts_tablet_manager.cc:1428] T 362a5f2576704a9295784c42fdd3e7ea P 6c404a3267bc4a11a6850d91dec7b21d: Time spent starting tablet: real 0.023s user 0.013s sys 0.004s
W20260812 02:36:57.633674 16438 tablet.cc:2369] T 362a5f2576704a9295784c42fdd3e7ea P ed2e8f19bfd642ac95e6b32d054ce8f4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 02:36:57.655072 16783 tablet.cc:2369] T 362a5f2576704a9295784c42fdd3e7ea P 6c404a3267bc4a11a6850d91dec7b21d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 02:36:57.691493 16831 raft_consensus.cc:482] T 362a5f2576704a9295784c42fdd3e7ea P ed2e8f19bfd642ac95e6b32d054ce8f4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 02:36:57.691848 16831 raft_consensus.cc:504] T 362a5f2576704a9295784c42fdd3e7ea P ed2e8f19bfd642ac95e6b32d054ce8f4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed2e8f19bfd642ac95e6b32d054ce8f4" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 34169 } } peers { permanent_uuid: "6c404a3267bc4a11a6850d91dec7b21d" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 35367 } } peers { permanent_uuid: "07b5778ae04246ec9f152a523a1ef33f" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 36973 } }
I20260812 02:36:57.694036 16831 leader_election.cc:288] T 362a5f2576704a9295784c42fdd3e7ea P ed2e8f19bfd642ac95e6b32d054ce8f4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c404a3267bc4a11a6850d91dec7b21d (127.7.132.131:35367), 07b5778ae04246ec9f152a523a1ef33f (127.7.132.130:36973)
I20260812 02:36:57.703488 16824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:57.694630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:52583 (local address 127.7.132.130:36973)
0812 02:36:57.694850 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:57.694857 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:57.696032 (+ 1175us) server_negotiation.cc:407] Connection header received
0812 02:36:57.696188 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:57.696193 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:57.696267 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:57.696348 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:36:57.697065 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:57.697918 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:57.701175 (+ 3257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:57.702542 (+ 1367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:57.702695 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:57.702768 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:57.703043 (+ 275us) server_negotiation.cc:299] Negotiation successful
0812 02:36:57.703216 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260812 02:36:57.703665 16834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:57.694485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.130:36973 (local address 127.7.132.129:52583)
0812 02:36:57.695904 (+ 1419us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:57.695925 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:57.696057 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:57.696468 (+ 411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:57.696473 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:57.696493 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:36:57.696898 (+ 405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:57.696909 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:57.698062 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:57.698070 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:57.701001 (+ 2931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:57.701017 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:57.702741 (+ 1724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:57.702745 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:57.702868 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:57.702893 (+ 25us) client_negotiation.cc:769] Sending connection context
0812 02:36:57.703009 (+ 116us) client_negotiation.cc:240] Negotiation successful
0812 02:36:57.703145 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1283,"mutex_wait_us":213,"thread_start_us":704,"threads_started":1}
I20260812 02:36:57.704711 16551 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "362a5f2576704a9295784c42fdd3e7ea" candidate_uuid: "ed2e8f19bfd642ac95e6b32d054ce8f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b5778ae04246ec9f152a523a1ef33f" is_pre_election: true
I20260812 02:36:57.705250 16551 raft_consensus.cc:2454] T 362a5f2576704a9295784c42fdd3e7ea P 07b5778ae04246ec9f152a523a1ef33f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ed2e8f19bfd642ac95e6b32d054ce8f4 in term 0.
I20260812 02:36:57.705470 16822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:57.694739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:39685 (local address 127.7.132.131:35367)
0812 02:36:57.694928 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:57.694934 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:57.695668 (+ 734us) server_negotiation.cc:407] Connection header received
0812 02:36:57.695990 (+ 322us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:57.695997 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:57.696083 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:57.696177 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:36:57.696896 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:57.698024 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:57.702883 (+ 4859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:57.704151 (+ 1268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:57.704327 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:57.704405 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:57.705075 (+ 670us) server_negotiation.cc:299] Negotiation successful
0812 02:36:57.705239 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 02:36:57.706022 16300 leader_election.cc:302] T 362a5f2576704a9295784c42fdd3e7ea P ed2e8f19bfd642ac95e6b32d054ce8f4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07b5778ae04246ec9f152a523a1ef33f, ed2e8f19bfd642ac95e6b32d054ce8f4; no voters:
I20260812 02:36:57.706313 16835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:57.694271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.131:35367 (local address 127.7.132.129:39685)
0812 02:36:57.695557 (+ 1286us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:57.695576 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:57.695803 (+ 227us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:57.696219 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:57.696223 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:57.696247 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:36:57.696735 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:57.696747 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:57.698173 (+ 1426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:57.698178 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:57.701001 (+ 2823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:57.701017 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:57.704693 (+ 3676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:57.704707 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:57.704880 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:57.704915 (+ 35us) client_negotiation.cc:769] Sending connection context
0812 02:36:57.705700 (+ 785us) client_negotiation.cc:240] Negotiation successful
0812 02:36:57.705839 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1115,"mutex_wait_us":244,"thread_start_us":1312,"threads_started":1}
I20260812 02:36:57.706612 16831 raft_consensus.cc:2781] T 362a5f2576704a9295784c42fdd3e7ea P ed2e8f19bfd642ac95e6b32d054ce8f4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 02:36:57.706765 16831 raft_consensus.cc:482] T 362a5f2576704a9295784c42fdd3e7ea P ed2e8f19bfd642ac95e6b32d054ce8f4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 02:36:57.706895 16831 raft_consensus.cc:3037] T 362a5f2576704a9295784c42fdd3e7ea P ed2e8f19bfd642ac95e6b32d054ce8f4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:36:57.707350 16724 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "362a5f2576704a9295784c42fdd3e7ea" candidate_uuid: "ed2e8f19bfd642ac95e6b32d054ce8f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c404a3267bc4a11a6850d91dec7b21d" is_pre_election: true
I20260812 02:36:57.707895 16724 raft_consensus.cc:2454] T 362a5f2576704a9295784c42fdd3e7ea P 6c404a3267bc4a11a6850d91dec7b21d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ed2e8f19bfd642ac95e6b32d054ce8f4 in term 0.
I20260812 02:36:57.709830 16831 raft_consensus.cc:504] T 362a5f2576704a9295784c42fdd3e7ea P ed2e8f19bfd642ac95e6b32d054ce8f4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed2e8f19bfd642ac95e6b32d054ce8f4" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 34169 } } peers { permanent_uuid: "6c404a3267bc4a11a6850d91dec7b21d" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 35367 } } peers { permanent_uuid: "07b5778ae04246ec9f152a523a1ef33f" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 36973 } }
I20260812 02:36:57.710525 16831 leader_election.cc:288] T 362a5f2576704a9295784c42fdd3e7ea P ed2e8f19bfd642ac95e6b32d054ce8f4 [CANDIDATE]: Term 1 election: Requested vote from peers 6c404a3267bc4a11a6850d91dec7b21d (127.7.132.131:35367), 07b5778ae04246ec9f152a523a1ef33f (127.7.132.130:36973)
I20260812 02:36:57.710924 16724 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "362a5f2576704a9295784c42fdd3e7ea" candidate_uuid: "ed2e8f19bfd642ac95e6b32d054ce8f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c404a3267bc4a11a6850d91dec7b21d"
I20260812 02:36:57.711072 16551 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "362a5f2576704a9295784c42fdd3e7ea" candidate_uuid: "ed2e8f19bfd642ac95e6b32d054ce8f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b5778ae04246ec9f152a523a1ef33f"
I20260812 02:36:57.711158 16724 raft_consensus.cc:3037] T 362a5f2576704a9295784c42fdd3e7ea P 6c404a3267bc4a11a6850d91dec7b21d [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:36:57.711303 16551 raft_consensus.cc:3037] T 362a5f2576704a9295784c42fdd3e7ea P 07b5778ae04246ec9f152a523a1ef33f [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:36:57.714602 16724 raft_consensus.cc:2454] T 362a5f2576704a9295784c42fdd3e7ea P 6c404a3267bc4a11a6850d91dec7b21d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed2e8f19bfd642ac95e6b32d054ce8f4 in term 1.
I20260812 02:36:57.714640 16551 raft_consensus.cc:2454] T 362a5f2576704a9295784c42fdd3e7ea P 07b5778ae04246ec9f152a523a1ef33f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed2e8f19bfd642ac95e6b32d054ce8f4 in term 1.
I20260812 02:36:57.715037 16297 leader_election.cc:302] T 362a5f2576704a9295784c42fdd3e7ea P ed2e8f19bfd642ac95e6b32d054ce8f4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c404a3267bc4a11a6850d91dec7b21d, ed2e8f19bfd642ac95e6b32d054ce8f4; no voters:
I20260812 02:36:57.715380 16831 raft_consensus.cc:2781] T 362a5f2576704a9295784c42fdd3e7ea P ed2e8f19bfd642ac95e6b32d054ce8f4 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:36:57.715862 16831 raft_consensus.cc:686] T 362a5f2576704a9295784c42fdd3e7ea P ed2e8f19bfd642ac95e6b32d054ce8f4 [term 1 LEADER]: Becoming Leader. State: Replica: ed2e8f19bfd642ac95e6b32d054ce8f4, State: Running, Role: LEADER
I20260812 02:36:57.716413 16831 consensus_queue.cc:237] T 362a5f2576704a9295784c42fdd3e7ea P ed2e8f19bfd642ac95e6b32d054ce8f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed2e8f19bfd642ac95e6b32d054ce8f4" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 34169 } } peers { permanent_uuid: "6c404a3267bc4a11a6850d91dec7b21d" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 35367 } } peers { permanent_uuid: "07b5778ae04246ec9f152a523a1ef33f" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 36973 } }
I20260812 02:36:57.722755 15222 catalog_manager.cc:5495] T 362a5f2576704a9295784c42fdd3e7ea P ed2e8f19bfd642ac95e6b32d054ce8f4 reported cstate change: term changed from 0 to 1, leader changed from <none> to ed2e8f19bfd642ac95e6b32d054ce8f4 (127.7.132.129). New cstate: current_term: 1 leader_uuid: "ed2e8f19bfd642ac95e6b32d054ce8f4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed2e8f19bfd642ac95e6b32d054ce8f4" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 34169 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c404a3267bc4a11a6850d91dec7b21d" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 35367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07b5778ae04246ec9f152a523a1ef33f" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 36973 } health_report { overall_health: UNKNOWN } } }
W20260812 02:36:57.747216 16611 tablet.cc:2369] T 362a5f2576704a9295784c42fdd3e7ea P 07b5778ae04246ec9f152a523a1ef33f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 02:36:57.758646 16819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:57.753137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55868 (local address 127.7.132.129:34169)
0812 02:36:57.753334 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:57.753340 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:57.753358 (+ 18us) server_negotiation.cc:407] Connection header received
0812 02:36:57.753554 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:57.753559 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:57.753640 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:57.753794 (+ 154us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 02:36:57.754277 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:57.755005 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:57.756393 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:57.756826 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:57.756968 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:57.757509 (+ 541us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 02:36:57.758000 (+ 491us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 02:36:57.758218 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:57.758300 (+ 82us) server_negotiation.cc:299] Negotiation successful
0812 02:36:57.758383 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 02:36:57.767004 16724 raft_consensus.cc:1261] T 362a5f2576704a9295784c42fdd3e7ea P 6c404a3267bc4a11a6850d91dec7b21d [term 1 FOLLOWER]: Refusing update from remote peer ed2e8f19bfd642ac95e6b32d054ce8f4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:36:57.767035 16551 raft_consensus.cc:1261] T 362a5f2576704a9295784c42fdd3e7ea P 07b5778ae04246ec9f152a523a1ef33f [term 1 FOLLOWER]: Refusing update from remote peer ed2e8f19bfd642ac95e6b32d054ce8f4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:36:57.768056 16831 consensus_queue.cc:1035] T 362a5f2576704a9295784c42fdd3e7ea P ed2e8f19bfd642ac95e6b32d054ce8f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07b5778ae04246ec9f152a523a1ef33f" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 36973 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:36:57.768361 16836 consensus_queue.cc:1035] T 362a5f2576704a9295784c42fdd3e7ea P ed2e8f19bfd642ac95e6b32d054ce8f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c404a3267bc4a11a6850d91dec7b21d" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 35367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:36:57.794252 16841 mvcc.cc:205] Tried to move back new op lower bound from 7317513083962646528 to 7317513083770216448. Current Snapshot: MvccSnapshot[applied={T|T < 7317513083962646528}]
I20260812 02:36:57.813746 15223 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33688:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 02:36:57.814030 15223 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33688:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 02:36:57.827210 15223 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 82e54874d347460c85c36da26b3bb8c8: Sending DeleteTablet for 3 replicas of tablet 362a5f2576704a9295784c42fdd3e7ea
I20260812 02:36:57.828657 16700 tablet_service.cc:1514] Processing DeleteTablet for tablet 362a5f2576704a9295784c42fdd3e7ea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:36:57 UTC) from {username='oryx', principal='oryx/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:37480
I20260812 02:36:57.828816 16353 tablet_service.cc:1514] Processing DeleteTablet for tablet 362a5f2576704a9295784c42fdd3e7ea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:36:57 UTC) from {username='oryx', principal='oryx/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:55858
I20260812 02:36:57.829595 16530 tablet_service.cc:1514] Processing DeleteTablet for tablet 362a5f2576704a9295784c42fdd3e7ea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:36:57 UTC) from {username='oryx', principal='oryx/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:58164
I20260812 02:36:57.832966 16853 tablet_replica.cc:330] T 362a5f2576704a9295784c42fdd3e7ea P 07b5778ae04246ec9f152a523a1ef33f: stopping tablet replica
I20260812 02:36:57.833514 16853 raft_consensus.cc:2229] T 362a5f2576704a9295784c42fdd3e7ea P 07b5778ae04246ec9f152a523a1ef33f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 02:36:57.834241 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 15180
I20260812 02:36:57.835598 15259 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:36:57.840332 16852 tablet_replica.cc:330] T 362a5f2576704a9295784c42fdd3e7ea P ed2e8f19bfd642ac95e6b32d054ce8f4: stopping tablet replica
I20260812 02:36:57.840960 16852 raft_consensus.cc:2229] T 362a5f2576704a9295784c42fdd3e7ea P ed2e8f19bfd642ac95e6b32d054ce8f4 [term 1 LEADER]: Raft consensus shutting down.
I20260812 02:36:57.841838 16852 raft_consensus.cc:2258] T 362a5f2576704a9295784c42fdd3e7ea P ed2e8f19bfd642ac95e6b32d054ce8f4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:36:57.844298 16852 ts_tablet_manager.cc:1905] T 362a5f2576704a9295784c42fdd3e7ea P ed2e8f19bfd642ac95e6b32d054ce8f4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:36:57.846886 16853 raft_consensus.cc:2258] T 362a5f2576704a9295784c42fdd3e7ea P 07b5778ae04246ec9f152a523a1ef33f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:36:57.849486 16853 ts_tablet_manager.cc:1905] T 362a5f2576704a9295784c42fdd3e7ea P 07b5778ae04246ec9f152a523a1ef33f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:36:57.851837 16851 tablet_replica.cc:330] T 362a5f2576704a9295784c42fdd3e7ea P 6c404a3267bc4a11a6850d91dec7b21d: stopping tablet replica
I20260812 02:36:57.852303 16851 raft_consensus.cc:2229] T 362a5f2576704a9295784c42fdd3e7ea P 6c404a3267bc4a11a6850d91dec7b21d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 02:36:57.852895 16851 raft_consensus.cc:2258] T 362a5f2576704a9295784c42fdd3e7ea P 6c404a3267bc4a11a6850d91dec7b21d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:36:57.866389 16851 ts_tablet_manager.cc:1905] T 362a5f2576704a9295784c42fdd3e7ea P 6c404a3267bc4a11a6850d91dec7b21d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:36:57.868876 16853 ts_tablet_manager.cc:1918] T 362a5f2576704a9295784c42fdd3e7ea P 07b5778ae04246ec9f152a523a1ef33f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 02:36:57.869077 16853 log.cc:1196] T 362a5f2576704a9295784c42fdd3e7ea P 07b5778ae04246ec9f152a523a1ef33f: Deleting WAL directory at /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/ts-1/wal/wals/362a5f2576704a9295784c42fdd3e7ea
I20260812 02:36:57.869659 16853 ts_tablet_manager.cc:1939] T 362a5f2576704a9295784c42fdd3e7ea P 07b5778ae04246ec9f152a523a1ef33f: Deleting consensus metadata
I20260812 02:36:57.873607 16851 ts_tablet_manager.cc:1918] T 362a5f2576704a9295784c42fdd3e7ea P 6c404a3267bc4a11a6850d91dec7b21d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 02:36:57.873795 16851 log.cc:1196] T 362a5f2576704a9295784c42fdd3e7ea P 6c404a3267bc4a11a6850d91dec7b21d: Deleting WAL directory at /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/ts-2/wal/wals/362a5f2576704a9295784c42fdd3e7ea
I20260812 02:36:57.854024 16852 ts_tablet_manager.cc:1918] T 362a5f2576704a9295784c42fdd3e7ea P ed2e8f19bfd642ac95e6b32d054ce8f4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 02:36:57.874370 16852 log.cc:1196] T 362a5f2576704a9295784c42fdd3e7ea P ed2e8f19bfd642ac95e6b32d054ce8f4: Deleting WAL directory at /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/ts-0/wal/wals/362a5f2576704a9295784c42fdd3e7ea
I20260812 02:36:57.874689 16797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:57.831387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41050 (local address 127.7.132.189:38815)
0812 02:36:57.831575 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:57.831582 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:57.835000 (+ 3418us) server_negotiation.cc:407] Connection header received
0812 02:36:57.835583 (+ 583us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:57.835590 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:57.835717 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:57.835799 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:57.848157 (+ 12358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:57.849677 (+ 1520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:57.856001 (+ 6324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:57.856610 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:57.856771 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:57.864196 (+ 7425us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:57.864203 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:57.864209 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:57.864256 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:57.868074 (+ 3818us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:57.869000 (+ 926us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:57.869006 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:57.869011 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:57.869172 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:57.869674 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:57.869679 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:57.869683 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:57.870177 (+ 494us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:57.870417 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:57.874122 (+ 3705us) server_negotiation.cc:299] Negotiation successful
0812 02:36:57.874373 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 02:36:57.874997 16852 ts_tablet_manager.cc:1939] T 362a5f2576704a9295784c42fdd3e7ea P ed2e8f19bfd642ac95e6b32d054ce8f4: Deleting consensus metadata
I20260812 02:36:57.883953 16851 ts_tablet_manager.cc:1939] T 362a5f2576704a9295784c42fdd3e7ea P 6c404a3267bc4a11a6850d91dec7b21d: Deleting consensus metadata
I20260812 02:36:57.894906 16800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:57.833294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35126 (local address 127.7.132.188:39485)
0812 02:36:57.833506 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:57.833515 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:57.871681 (+ 38166us) server_negotiation.cc:407] Connection header received
0812 02:36:57.871965 (+ 284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:57.871972 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:57.872060 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:57.872137 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:57.875270 (+ 3133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:57.876351 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:57.881633 (+ 5282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:57.882477 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:57.882593 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:57.887884 (+ 5291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:57.887893 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:57.887899 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:57.887950 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:57.891733 (+ 3783us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:57.892604 (+ 871us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:57.892610 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:57.892614 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:57.892774 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:57.893223 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:57.893228 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:57.893232 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:57.893631 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:57.893915 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:57.894297 (+ 382us) server_negotiation.cc:299] Negotiation successful
0812 02:36:57.894574 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260812 02:36:57.995247 15208 catalog_manager.cc:4841] TS ed2e8f19bfd642ac95e6b32d054ce8f4 (127.7.132.129:34169): tablet 362a5f2576704a9295784c42fdd3e7ea (table test-table [id=da33c73d3e09419eb75e4422507f1d00]) successfully deleted
I20260812 02:36:57.995410 15205 catalog_manager.cc:4841] TS 6c404a3267bc4a11a6850d91dec7b21d (127.7.132.131:35367): tablet 362a5f2576704a9295784c42fdd3e7ea (table test-table [id=da33c73d3e09419eb75e4422507f1d00]) successfully deleted
I20260812 02:36:57.995874 15208 catalog_manager.cc:4841] TS 07b5778ae04246ec9f152a523a1ef33f (127.7.132.130:36973): tablet 362a5f2576704a9295784c42fdd3e7ea (table test-table [id=da33c73d3e09419eb75e4422507f1d00]) successfully deleted
I20260812 02:36:57.996199 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 15180
I20260812 02:36:58.047851 16797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:58.028121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41052 (local address 127.7.132.189:38815)
0812 02:36:58.028395 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:58.028405 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:58.029443 (+ 1038us) server_negotiation.cc:407] Connection header received
0812 02:36:58.030703 (+ 1260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:58.030712 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:58.030828 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:58.030923 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:58.033118 (+ 2195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:58.033900 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:58.035798 (+ 1898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:58.036428 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:58.036584 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:58.040596 (+ 4012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:58.040604 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:58.040609 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:58.040652 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:58.044349 (+ 3697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:58.045079 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:58.045085 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:58.045088 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:58.045232 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:58.045750 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:58.045757 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:58.045761 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:58.046072 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:58.046417 (+ 345us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:58.047220 (+ 803us) server_negotiation.cc:299] Negotiation successful
0812 02:36:58.047458 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260812 02:36:58.049480 16800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:58.029542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35132 (local address 127.7.132.188:39485)
0812 02:36:58.030851 (+ 1309us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:58.030859 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:58.030880 (+ 21us) server_negotiation.cc:407] Connection header received
0812 02:36:58.030947 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:58.030952 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:58.031035 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:58.031140 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:58.033904 (+ 2764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:58.034721 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:58.036259 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:58.036880 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:58.037019 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:58.041496 (+ 4477us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:58.041503 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:58.041508 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:58.041566 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:58.045343 (+ 3777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:58.046735 (+ 1392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:58.046741 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:58.046745 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:58.046948 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:58.047389 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:58.047395 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:58.047399 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:58.047777 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:58.048366 (+ 589us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:58.048700 (+ 334us) server_negotiation.cc:299] Negotiation successful
0812 02:36:58.049088 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1163}
I20260812 02:36:58.800316 16863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:58.799508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:33723 (local address 127.7.132.131:42783)
0812 02:36:58.799981 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:58.800118 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:33723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":218,"threads_started":1}
W20260812 02:36:58.800796 16780 heartbeater.cc:644] Failed to heartbeat to 127.7.132.190:33723 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.7.132.190:33723: connect: Connection refused (error 111)
I20260812 02:36:58.804049 16864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:58.803202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:33723 (local address 127.7.132.130:38703)
0812 02:36:58.803686 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:58.803838 (+ 152us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:33723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":260,"threads_started":1}
W20260812 02:36:58.804486 16609 heartbeater.cc:644] Failed to heartbeat to 127.7.132.190:33723 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.7.132.190:33723: connect: Connection refused (error 111)
I20260812 02:36:58.809065 16866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:58.808201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:33723 (local address 127.7.132.129:47575)
0812 02:36:58.808707 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:58.808862 (+ 155us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:33723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":265,"threads_started":1}
W20260812 02:36:58.809451 16436 heartbeater.cc:644] Failed to heartbeat to 127.7.132.190:33723 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.7.132.190:33723: connect: Connection refused (error 111)
I20260812 02:36:59.325311 16868 raft_consensus.cc:482] T 00000000000000000000000000000000 P 351974fae6dd4c168a844a01b53d9927 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 82e54874d347460c85c36da26b3bb8c8)
I20260812 02:36:59.325527 16868 raft_consensus.cc:504] T 00000000000000000000000000000000 P 351974fae6dd4c168a844a01b53d9927 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "82e54874d347460c85c36da26b3bb8c8" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 33723 } } peers { permanent_uuid: "351974fae6dd4c168a844a01b53d9927" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 38815 } attrs { promote: false } } peers { permanent_uuid: "a6d878afb46e4df18827ac19aa9f97bd" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 39485 } attrs { promote: false } }
I20260812 02:36:59.326508 16868 leader_election.cc:288] T 00000000000000000000000000000000 P 351974fae6dd4c168a844a01b53d9927 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 82e54874d347460c85c36da26b3bb8c8 (127.7.132.190:33723), a6d878afb46e4df18827ac19aa9f97bd (127.7.132.188:39485)
I20260812 02:36:59.326555 16869 raft_consensus.cc:482] T 00000000000000000000000000000000 P a6d878afb46e4df18827ac19aa9f97bd [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 82e54874d347460c85c36da26b3bb8c8)
I20260812 02:36:59.326915 16869 raft_consensus.cc:504] T 00000000000000000000000000000000 P a6d878afb46e4df18827ac19aa9f97bd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "82e54874d347460c85c36da26b3bb8c8" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 33723 } } peers { permanent_uuid: "351974fae6dd4c168a844a01b53d9927" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 38815 } attrs { promote: false } } peers { permanent_uuid: "a6d878afb46e4df18827ac19aa9f97bd" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 39485 } attrs { promote: false } }
I20260812 02:36:59.327838 16870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:59.326763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:33723 (local address 127.0.0.1:34082)
0812 02:36:59.327280 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:59.327545 (+ 265us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:33723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":225,"threads_started":1}
W20260812 02:36:59.329239 15965 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.132.190:33723: connect: Connection refused (error 111)
I20260812 02:36:59.330256 16875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:59.328986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:33723 (local address 127.0.0.1:34084)
0812 02:36:59.329526 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:59.329743 (+ 217us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:33723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":243,"threads_started":1}
W20260812 02:36:59.330754 16192 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.132.190:33723: connect: Connection refused (error 111)
I20260812 02:36:59.330899 16869 leader_election.cc:288] T 00000000000000000000000000000000 P a6d878afb46e4df18827ac19aa9f97bd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 82e54874d347460c85c36da26b3bb8c8 (127.7.132.190:33723), 351974fae6dd4c168a844a01b53d9927 (127.7.132.189:38815)
I20260812 02:36:59.336622 16870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:59.327408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:39485 (local address 127.0.0.1:35138)
0812 02:36:59.328081 (+ 673us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:59.328105 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:59.328288 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:59.329314 (+ 1026us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:59.329321 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:59.329377 (+ 56us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:36:59.329654 (+ 277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:59.329670 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:59.331591 (+ 1921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:59.331599 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:59.333948 (+ 2349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:59.333964 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:59.335752 (+ 1788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:59.335759 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:59.335964 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:59.336000 (+ 36us) client_negotiation.cc:769] Sending connection context
0812 02:36:59.336073 (+ 73us) client_negotiation.cc:240] Negotiation successful
0812 02:36:59.336215 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":556,"mutex_wait_us":1,"thread_start_us":1506,"threads_started":1}
I20260812 02:36:59.337541 16873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:59.327681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35138 (local address 127.7.132.188:39485)
0812 02:36:59.328894 (+ 1213us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:59.328903 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:59.328926 (+ 23us) server_negotiation.cc:407] Connection header received
0812 02:36:59.329006 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:59.329013 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:59.329108 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:59.329807 (+ 699us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:36:59.330049 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:59.331378 (+ 1329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:59.334166 (+ 2788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:59.335539 (+ 1373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:59.336860 (+ 1321us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:59.336980 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:59.337037 (+ 57us) server_negotiation.cc:299] Negotiation successful
0812 02:36:59.337175 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1071,"thread_start_us":917,"threads_started":1}
I20260812 02:36:59.338847 16228 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "351974fae6dd4c168a844a01b53d9927" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "a6d878afb46e4df18827ac19aa9f97bd" is_pre_election: true
I20260812 02:36:59.339287 16228 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a6d878afb46e4df18827ac19aa9f97bd [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 351974fae6dd4c168a844a01b53d9927 in term 1.
I20260812 02:36:59.339638 16875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:59.329855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.189:38815 (local address 127.0.0.1:41058)
0812 02:36:59.330587 (+ 732us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:59.330604 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:59.330710 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:59.332248 (+ 1538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:59.332255 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:59.332284 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:36:59.332598 (+ 314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:59.332613 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:59.334407 (+ 1794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:59.334413 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:59.336471 (+ 2058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:59.336486 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:59.338708 (+ 2222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:59.338715 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:59.338969 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:59.339025 (+ 56us) client_negotiation.cc:769] Sending connection context
0812 02:36:59.339144 (+ 119us) client_negotiation.cc:240] Negotiation successful
0812 02:36:59.339302 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":640,"thread_start_us":1397,"threads_started":1}
I20260812 02:36:59.340431 16878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:59.330245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41058 (local address 127.7.132.189:38815)
0812 02:36:59.331912 (+ 1667us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:59.331919 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:59.331944 (+ 25us) server_negotiation.cc:407] Connection header received
0812 02:36:59.332015 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:59.332021 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:59.332084 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:59.332748 (+ 664us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:36:59.333036 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:59.334261 (+ 1225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:59.336916 (+ 2655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:59.338501 (+ 1585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:59.338650 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:59.338760 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:59.339854 (+ 1094us) server_negotiation.cc:299] Negotiation successful
0812 02:36:59.339991 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1552,"thread_start_us":1312,"threads_started":1}
I20260812 02:36:59.340497 16870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:59.340015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:33723 (local address 127.0.0.1:34104)
0812 02:36:59.340190 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:59.340305 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:33723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260812 02:36:59.340690 16875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:59.340246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:33723 (local address 127.0.0.1:34116)
0812 02:36:59.340437 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:59.340529 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:33723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260812 02:36:59.340855 15965 leader_election.cc:302] T 00000000000000000000000000000000 P 351974fae6dd4c168a844a01b53d9927 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 351974fae6dd4c168a844a01b53d9927, a6d878afb46e4df18827ac19aa9f97bd; no voters:
W20260812 02:36:59.341097 16192 leader_election.cc:334] T 00000000000000000000000000000000 P a6d878afb46e4df18827ac19aa9f97bd [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 82e54874d347460c85c36da26b3bb8c8 (127.7.132.190:33723): Network error: Client connection negotiation failed: client connection to 127.7.132.190:33723: connect: Connection refused (error 111)
I20260812 02:36:59.341204 16868 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 351974fae6dd4c168a844a01b53d9927 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260812 02:36:59.341250 15965 leader_election.cc:334] T 00000000000000000000000000000000 P 351974fae6dd4c168a844a01b53d9927 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 82e54874d347460c85c36da26b3bb8c8 (127.7.132.190:33723): Network error: Client connection negotiation failed: client connection to 127.7.132.190:33723: connect: Connection refused (error 111)
I20260812 02:36:59.341347 16868 raft_consensus.cc:482] T 00000000000000000000000000000000 P 351974fae6dd4c168a844a01b53d9927 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 82e54874d347460c85c36da26b3bb8c8)
I20260812 02:36:59.341280 15998 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a6d878afb46e4df18827ac19aa9f97bd" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "351974fae6dd4c168a844a01b53d9927" is_pre_election: true
I20260812 02:36:59.341478 16868 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 351974fae6dd4c168a844a01b53d9927 [term 1 FOLLOWER]: Advancing to term 2
I20260812 02:36:59.343962 16868 raft_consensus.cc:504] T 00000000000000000000000000000000 P 351974fae6dd4c168a844a01b53d9927 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "82e54874d347460c85c36da26b3bb8c8" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 33723 } } peers { permanent_uuid: "351974fae6dd4c168a844a01b53d9927" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 38815 } attrs { promote: false } } peers { permanent_uuid: "a6d878afb46e4df18827ac19aa9f97bd" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 39485 } attrs { promote: false } }
I20260812 02:36:59.344435 15998 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 351974fae6dd4c168a844a01b53d9927 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a6d878afb46e4df18827ac19aa9f97bd in current term 2: Already voted for candidate 351974fae6dd4c168a844a01b53d9927 in this term.
I20260812 02:36:59.344713 16868 leader_election.cc:288] T 00000000000000000000000000000000 P 351974fae6dd4c168a844a01b53d9927 [CANDIDATE]: Term 2 election: Requested vote from peers 82e54874d347460c85c36da26b3bb8c8 (127.7.132.190:33723), a6d878afb46e4df18827ac19aa9f97bd (127.7.132.188:39485)
I20260812 02:36:59.345446 16870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:59.345026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:33723 (local address 127.0.0.1:34130)
0812 02:36:59.345189 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:59.345284 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:33723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260812 02:36:59.345492 16194 leader_election.cc:302] T 00000000000000000000000000000000 P a6d878afb46e4df18827ac19aa9f97bd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a6d878afb46e4df18827ac19aa9f97bd; no voters: 351974fae6dd4c168a844a01b53d9927, 82e54874d347460c85c36da26b3bb8c8
I20260812 02:36:59.346130 16869 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a6d878afb46e4df18827ac19aa9f97bd [term 1 FOLLOWER]: Advancing to term 2
I20260812 02:36:59.346320 16228 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "351974fae6dd4c168a844a01b53d9927" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "a6d878afb46e4df18827ac19aa9f97bd"
I20260812 02:36:59.347729 16870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:59.347221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:33723 (local address 127.0.0.1:34138)
0812 02:36:59.347442 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:59.347539 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:33723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260812 02:36:59.347908 15965 leader_election.cc:334] T 00000000000000000000000000000000 P 351974fae6dd4c168a844a01b53d9927 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 82e54874d347460c85c36da26b3bb8c8 (127.7.132.190:33723): Network error: Client connection negotiation failed: client connection to 127.7.132.190:33723: connect: Connection refused (error 111)
I20260812 02:36:59.349063 16869 raft_consensus.cc:2726] T 00000000000000000000000000000000 P a6d878afb46e4df18827ac19aa9f97bd [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 02:36:59.352954 16228 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a6d878afb46e4df18827ac19aa9f97bd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 351974fae6dd4c168a844a01b53d9927 in term 2.
I20260812 02:36:59.353792 15965 leader_election.cc:302] T 00000000000000000000000000000000 P 351974fae6dd4c168a844a01b53d9927 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 351974fae6dd4c168a844a01b53d9927, a6d878afb46e4df18827ac19aa9f97bd; no voters: 82e54874d347460c85c36da26b3bb8c8
I20260812 02:36:59.354100 16868 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 351974fae6dd4c168a844a01b53d9927 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 02:36:59.355029 16868 raft_consensus.cc:686] T 00000000000000000000000000000000 P 351974fae6dd4c168a844a01b53d9927 [term 2 LEADER]: Becoming Leader. State: Replica: 351974fae6dd4c168a844a01b53d9927, State: Running, Role: LEADER
I20260812 02:36:59.355546 16868 consensus_queue.cc:237] T 00000000000000000000000000000000 P 351974fae6dd4c168a844a01b53d9927 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "82e54874d347460c85c36da26b3bb8c8" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 33723 } } peers { permanent_uuid: "351974fae6dd4c168a844a01b53d9927" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 38815 } attrs { promote: false } } peers { permanent_uuid: "a6d878afb46e4df18827ac19aa9f97bd" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 39485 } attrs { promote: false } }
I20260812 02:36:59.358059 16880 sys_catalog.cc:455] T 00000000000000000000000000000000 P 351974fae6dd4c168a844a01b53d9927 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 351974fae6dd4c168a844a01b53d9927. Latest consensus state: current_term: 2 leader_uuid: "351974fae6dd4c168a844a01b53d9927" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "82e54874d347460c85c36da26b3bb8c8" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 33723 } } peers { permanent_uuid: "351974fae6dd4c168a844a01b53d9927" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 38815 } attrs { promote: false } } peers { permanent_uuid: "a6d878afb46e4df18827ac19aa9f97bd" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 39485 } attrs { promote: false } } }
I20260812 02:36:59.358647 16880 sys_catalog.cc:458] T 00000000000000000000000000000000 P 351974fae6dd4c168a844a01b53d9927 [sys.catalog]: This master's current role is: LEADER
I20260812 02:36:59.359469 16882 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:36:59.364089 16882 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 02:36:59.364974 16882 catalog_manager.cc:1259] Loaded cluster ID: eb5dad367dc04d1ba16a1cfb4c2dc8a7
I20260812 02:36:59.365051 16882 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 02:36:59.366302 16882 catalog_manager.cc:1504] Loading token signing keys...
I20260812 02:36:59.367132 16882 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 351974fae6dd4c168a844a01b53d9927: Loaded TSK: 0
I20260812 02:36:59.368011 16882 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 02:36:59.396668 15984 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41052:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 02:36:59.405910 15984 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 02:36:59.410727 16870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:59.410331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:33723 (local address 127.0.0.1:34154)
0812 02:36:59.410492 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:59.410592 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:33723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260812 02:36:59.411227 16228 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a6d878afb46e4df18827ac19aa9f97bd [term 2 FOLLOWER]: Refusing update from remote peer 351974fae6dd4c168a844a01b53d9927: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260812 02:36:59.411962 16870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:59.411490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:33723 (local address 127.0.0.1:34166)
0812 02:36:59.411665 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:59.411789 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:33723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260812 02:36:59.411991 16868 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 351974fae6dd4c168a844a01b53d9927 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6d878afb46e4df18827ac19aa9f97bd" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 39485 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260812 02:36:59.412374 15965 consensus_peers.cc:487] T 00000000000000000000000000000000 P 351974fae6dd4c168a844a01b53d9927 -> Peer 82e54874d347460c85c36da26b3bb8c8 (127.7.132.190:33723): Couldn't send request to peer 82e54874d347460c85c36da26b3bb8c8. Status: Network error: Client connection negotiation failed: client connection to 127.7.132.190:33723: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260812 02:36:59.414798 16869 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6d878afb46e4df18827ac19aa9f97bd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 351974fae6dd4c168a844a01b53d9927. Latest consensus state: current_term: 2 leader_uuid: "351974fae6dd4c168a844a01b53d9927" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "82e54874d347460c85c36da26b3bb8c8" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 33723 } } peers { permanent_uuid: "351974fae6dd4c168a844a01b53d9927" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 38815 } attrs { promote: false } } peers { permanent_uuid: "a6d878afb46e4df18827ac19aa9f97bd" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 39485 } attrs { promote: false } } }
I20260812 02:36:59.415108 16869 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6d878afb46e4df18827ac19aa9f97bd [sys.catalog]: This master's current role is: FOLLOWER
I20260812 02:36:59.416759 16880 sys_catalog.cc:455] T 00000000000000000000000000000000 P 351974fae6dd4c168a844a01b53d9927 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "351974fae6dd4c168a844a01b53d9927" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "82e54874d347460c85c36da26b3bb8c8" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 33723 } } peers { permanent_uuid: "351974fae6dd4c168a844a01b53d9927" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 38815 } attrs { promote: false } } peers { permanent_uuid: "a6d878afb46e4df18827ac19aa9f97bd" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 39485 } attrs { promote: false } } }
I20260812 02:36:59.417064 16880 sys_catalog.cc:458] T 00000000000000000000000000000000 P 351974fae6dd4c168a844a01b53d9927 [sys.catalog]: This master's current role is: LEADER
I20260812 02:36:59.417704 16869 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6d878afb46e4df18827ac19aa9f97bd [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "351974fae6dd4c168a844a01b53d9927" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "82e54874d347460c85c36da26b3bb8c8" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 33723 } } peers { permanent_uuid: "351974fae6dd4c168a844a01b53d9927" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 38815 } attrs { promote: false } } peers { permanent_uuid: "a6d878afb46e4df18827ac19aa9f97bd" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 39485 } attrs { promote: false } } }
I20260812 02:36:59.418032 16869 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6d878afb46e4df18827ac19aa9f97bd [sys.catalog]: This master's current role is: FOLLOWER
I20260812 02:36:59.435624 16870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:59.425340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.130:36973 (local address 127.0.0.1:58166)
0812 02:36:59.425744 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:59.425765 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:59.425875 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:59.426739 (+ 864us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:59.426747 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:59.426809 (+ 62us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:36:59.427118 (+ 309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:59.427142 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:59.429360 (+ 2218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:59.429370 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:59.432124 (+ 2754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:59.432159 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:59.434446 (+ 2287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:59.434455 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:59.434742 (+ 287us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:59.434808 (+ 66us) client_negotiation.cc:769] Sending connection context
0812 02:36:59.434900 (+ 92us) client_negotiation.cc:240] Negotiation successful
0812 02:36:59.435118 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"spinlock_wait_cycles":10368}
I20260812 02:36:59.437278 16890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:59.425746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58166 (local address 127.7.132.130:36973)
0812 02:36:59.426292 (+ 546us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:59.426302 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:59.426328 (+ 26us) server_negotiation.cc:407] Connection header received
0812 02:36:59.426422 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:59.426431 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:59.426533 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:59.427309 (+ 776us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:36:59.427536 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:59.429137 (+ 1601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:59.432444 (+ 3307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:59.434213 (+ 1769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:59.436587 (+ 2374us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:59.436725 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:59.436790 (+ 65us) server_negotiation.cc:299] Negotiation successful
0812 02:36:59.436966 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":391,"thread_start_us":265,"threads_started":1}
I20260812 02:36:59.438601 16530 tablet_service.cc:1467] Processing CreateTablet for tablet 88e1b260fa1740b8b9238b91f65053f3 (DEFAULT_TABLE table=test-table [id=ea6d5dbc4022425f9f352b57572c88a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:36:59.439126 16530 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88e1b260fa1740b8b9238b91f65053f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:36:59.443948 16891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:59.426410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37484 (local address 127.7.132.131:35367)
0812 02:36:59.437143 (+ 10733us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:59.437149 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:59.437171 (+ 22us) server_negotiation.cc:407] Connection header received
0812 02:36:59.437245 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:59.437254 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:59.437356 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:59.438188 (+ 832us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:36:59.438392 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:59.439519 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:59.441537 (+ 2018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:59.442785 (+ 1248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:59.442962 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:59.443082 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:59.443474 (+ 392us) server_negotiation.cc:299] Negotiation successful
0812 02:36:59.443648 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10612,"thread_start_us":10516,"threads_started":1}
I20260812 02:36:59.443965 16871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:59.426289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.131:35367 (local address 127.0.0.1:37484)
0812 02:36:59.426629 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:59.426653 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:59.426805 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:59.437534 (+ 10729us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:59.437542 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:59.437577 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:36:59.437997 (+ 420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:59.438009 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:59.439696 (+ 1687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:59.439701 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:59.441356 (+ 1655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:59.441370 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:59.443066 (+ 1696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:59.443073 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:59.443282 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:59.443320 (+ 38us) client_negotiation.cc:769] Sending connection context
0812 02:36:59.443426 (+ 106us) client_negotiation.cc:240] Negotiation successful
0812 02:36:59.443585 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"mutex_wait_us":84,"spinlock_wait_cycles":5504}
I20260812 02:36:59.444723 16700 tablet_service.cc:1467] Processing CreateTablet for tablet 88e1b260fa1740b8b9238b91f65053f3 (DEFAULT_TABLE table=test-table [id=ea6d5dbc4022425f9f352b57572c88a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:36:59.445189 16700 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88e1b260fa1740b8b9238b91f65053f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:36:59.446342 16894 tablet_bootstrap.cc:492] T 88e1b260fa1740b8b9238b91f65053f3 P 07b5778ae04246ec9f152a523a1ef33f: Bootstrap starting.
I20260812 02:36:59.448177 16894 tablet_bootstrap.cc:654] T 88e1b260fa1740b8b9238b91f65053f3 P 07b5778ae04246ec9f152a523a1ef33f: Neither blocks nor log segments found. Creating new log.
I20260812 02:36:59.449996 16870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:59.427389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:34169 (local address 127.0.0.1:55882)
0812 02:36:59.435964 (+ 8575us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:59.435990 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:59.436145 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:59.438299 (+ 2154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:59.438304 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:59.438333 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:36:59.438520 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:59.438530 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:59.439891 (+ 1361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:59.439896 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:59.441948 (+ 2052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:59.441964 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:59.443637 (+ 1673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:59.443643 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:59.443879 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:59.443913 (+ 34us) client_negotiation.cc:769] Sending connection context
0812 02:36:59.443995 (+ 82us) client_negotiation.cc:240] Negotiation successful
0812 02:36:59.444163 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8433,"thread_start_us":9147,"threads_started":1}
I20260812 02:36:59.450265 16896 tablet_bootstrap.cc:492] T 88e1b260fa1740b8b9238b91f65053f3 P 6c404a3267bc4a11a6850d91dec7b21d: Bootstrap starting.
I20260812 02:36:59.451505 16893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:59.427913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55882 (local address 127.7.132.129:34169)
0812 02:36:59.437890 (+ 9977us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:59.437897 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:59.437918 (+ 21us) server_negotiation.cc:407] Connection header received
0812 02:36:59.437997 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:59.438002 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:59.438122 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:59.438272 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:36:59.438721 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:59.439741 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:59.442169 (+ 2428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:59.443462 (+ 1293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:59.450815 (+ 7353us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:59.450931 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:59.450993 (+ 62us) server_negotiation.cc:299] Negotiation successful
0812 02:36:59.451195 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9836,"thread_start_us":9972,"threads_started":1}
I20260812 02:36:59.452046 16896 tablet_bootstrap.cc:654] T 88e1b260fa1740b8b9238b91f65053f3 P 6c404a3267bc4a11a6850d91dec7b21d: Neither blocks nor log segments found. Creating new log.
I20260812 02:36:59.452037 16894 tablet_bootstrap.cc:492] T 88e1b260fa1740b8b9238b91f65053f3 P 07b5778ae04246ec9f152a523a1ef33f: No bootstrap required, opened a new log
I20260812 02:36:59.452404 16353 tablet_service.cc:1467] Processing CreateTablet for tablet 88e1b260fa1740b8b9238b91f65053f3 (DEFAULT_TABLE table=test-table [id=ea6d5dbc4022425f9f352b57572c88a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:36:59.452757 16353 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88e1b260fa1740b8b9238b91f65053f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:36:59.453352 16894 ts_tablet_manager.cc:1397] T 88e1b260fa1740b8b9238b91f65053f3 P 07b5778ae04246ec9f152a523a1ef33f: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.003s
I20260812 02:36:59.454207 16894 raft_consensus.cc:348] T 88e1b260fa1740b8b9238b91f65053f3 P 07b5778ae04246ec9f152a523a1ef33f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b5778ae04246ec9f152a523a1ef33f" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 36973 } } peers { permanent_uuid: "ed2e8f19bfd642ac95e6b32d054ce8f4" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 34169 } } peers { permanent_uuid: "6c404a3267bc4a11a6850d91dec7b21d" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 35367 } }
I20260812 02:36:59.454444 16894 raft_consensus.cc:374] T 88e1b260fa1740b8b9238b91f65053f3 P 07b5778ae04246ec9f152a523a1ef33f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:36:59.454555 16894 raft_consensus.cc:729] T 88e1b260fa1740b8b9238b91f65053f3 P 07b5778ae04246ec9f152a523a1ef33f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07b5778ae04246ec9f152a523a1ef33f, State: Initialized, Role: FOLLOWER
I20260812 02:36:59.454818 16894 consensus_queue.cc:260] T 88e1b260fa1740b8b9238b91f65053f3 P 07b5778ae04246ec9f152a523a1ef33f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b5778ae04246ec9f152a523a1ef33f" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 36973 } } peers { permanent_uuid: "ed2e8f19bfd642ac95e6b32d054ce8f4" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 34169 } } peers { permanent_uuid: "6c404a3267bc4a11a6850d91dec7b21d" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 35367 } }
I20260812 02:36:59.455087 16896 tablet_bootstrap.cc:492] T 88e1b260fa1740b8b9238b91f65053f3 P 6c404a3267bc4a11a6850d91dec7b21d: No bootstrap required, opened a new log
I20260812 02:36:59.455267 16896 ts_tablet_manager.cc:1397] T 88e1b260fa1740b8b9238b91f65053f3 P 6c404a3267bc4a11a6850d91dec7b21d: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260812 02:36:59.456316 16896 raft_consensus.cc:348] T 88e1b260fa1740b8b9238b91f65053f3 P 6c404a3267bc4a11a6850d91dec7b21d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b5778ae04246ec9f152a523a1ef33f" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 36973 } } peers { permanent_uuid: "ed2e8f19bfd642ac95e6b32d054ce8f4" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 34169 } } peers { permanent_uuid: "6c404a3267bc4a11a6850d91dec7b21d" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 35367 } }
I20260812 02:36:59.456622 16896 raft_consensus.cc:374] T 88e1b260fa1740b8b9238b91f65053f3 P 6c404a3267bc4a11a6850d91dec7b21d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:36:59.456741 16896 raft_consensus.cc:729] T 88e1b260fa1740b8b9238b91f65053f3 P 6c404a3267bc4a11a6850d91dec7b21d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c404a3267bc4a11a6850d91dec7b21d, State: Initialized, Role: FOLLOWER
I20260812 02:36:59.457074 16896 consensus_queue.cc:260] T 88e1b260fa1740b8b9238b91f65053f3 P 6c404a3267bc4a11a6850d91dec7b21d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b5778ae04246ec9f152a523a1ef33f" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 36973 } } peers { permanent_uuid: "ed2e8f19bfd642ac95e6b32d054ce8f4" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 34169 } } peers { permanent_uuid: "6c404a3267bc4a11a6850d91dec7b21d" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 35367 } }
I20260812 02:36:59.459264 16896 ts_tablet_manager.cc:1428] T 88e1b260fa1740b8b9238b91f65053f3 P 6c404a3267bc4a11a6850d91dec7b21d: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260812 02:36:59.460438 16901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:59.459328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:33723 (local address 127.7.132.131:44527)
0812 02:36:59.459977 (+ 649us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:59.460124 (+ 147us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:33723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":346,"threads_started":1}
I20260812 02:36:59.464004 16781 heartbeater.cc:503] Master 127.7.132.189:38815 was elected leader, sending a full tablet report...
I20260812 02:36:59.464502 16900 tablet_bootstrap.cc:492] T 88e1b260fa1740b8b9238b91f65053f3 P ed2e8f19bfd642ac95e6b32d054ce8f4: Bootstrap starting.
I20260812 02:36:59.465790 16901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:59.465225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:33723 (local address 127.7.132.131:43139)
0812 02:36:59.465449 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:59.465576 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:33723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260812 02:36:59.467950 16900 tablet_bootstrap.cc:654] T 88e1b260fa1740b8b9238b91f65053f3 P ed2e8f19bfd642ac95e6b32d054ce8f4: Neither blocks nor log segments found. Creating new log.
I20260812 02:36:59.469380 16894 ts_tablet_manager.cc:1428] T 88e1b260fa1740b8b9238b91f65053f3 P 07b5778ae04246ec9f152a523a1ef33f: Time spent starting tablet: real 0.016s user 0.007s sys 0.007s
I20260812 02:36:59.471508 16903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:59.469646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:33723 (local address 127.7.132.130:45303)
0812 02:36:59.470300 (+ 654us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:59.471146 (+ 846us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:33723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":488,"thread_start_us":287,"threads_started":1}
I20260812 02:36:59.471817 16900 tablet_bootstrap.cc:492] T 88e1b260fa1740b8b9238b91f65053f3 P ed2e8f19bfd642ac95e6b32d054ce8f4: No bootstrap required, opened a new log
I20260812 02:36:59.472038 16900 ts_tablet_manager.cc:1397] T 88e1b260fa1740b8b9238b91f65053f3 P ed2e8f19bfd642ac95e6b32d054ce8f4: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20260812 02:36:59.472733 16610 heartbeater.cc:503] Master 127.7.132.189:38815 was elected leader, sending a full tablet report...
I20260812 02:36:59.473098 16900 raft_consensus.cc:348] T 88e1b260fa1740b8b9238b91f65053f3 P ed2e8f19bfd642ac95e6b32d054ce8f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b5778ae04246ec9f152a523a1ef33f" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 36973 } } peers { permanent_uuid: "ed2e8f19bfd642ac95e6b32d054ce8f4" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 34169 } } peers { permanent_uuid: "6c404a3267bc4a11a6850d91dec7b21d" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 35367 } }
I20260812 02:36:59.473400 16900 raft_consensus.cc:374] T 88e1b260fa1740b8b9238b91f65053f3 P ed2e8f19bfd642ac95e6b32d054ce8f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:36:59.473557 16900 raft_consensus.cc:729] T 88e1b260fa1740b8b9238b91f65053f3 P ed2e8f19bfd642ac95e6b32d054ce8f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed2e8f19bfd642ac95e6b32d054ce8f4, State: Initialized, Role: FOLLOWER
I20260812 02:36:59.473796 16903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:59.472220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:33723 (local address 127.7.132.130:60875)
0812 02:36:59.472469 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:59.473641 (+ 1172us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:33723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260812 02:36:59.473977 16900 consensus_queue.cc:260] T 88e1b260fa1740b8b9238b91f65053f3 P ed2e8f19bfd642ac95e6b32d054ce8f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b5778ae04246ec9f152a523a1ef33f" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 36973 } } peers { permanent_uuid: "ed2e8f19bfd642ac95e6b32d054ce8f4" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 34169 } } peers { permanent_uuid: "6c404a3267bc4a11a6850d91dec7b21d" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 35367 } }
I20260812 02:36:59.475919 16900 ts_tablet_manager.cc:1428] T 88e1b260fa1740b8b9238b91f65053f3 P ed2e8f19bfd642ac95e6b32d054ce8f4: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260812 02:36:59.476320 16905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:59.475180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:33723 (local address 127.7.132.129:38623)
0812 02:36:59.475813 (+ 633us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:59.476023 (+ 210us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:33723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":312,"threads_started":1}
I20260812 02:36:59.477224 16437 heartbeater.cc:503] Master 127.7.132.189:38815 was elected leader, sending a full tablet report...
I20260812 02:36:59.477402 16905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:59.476990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:33723 (local address 127.7.132.129:48549)
0812 02:36:59.477146 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:59.477250 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:33723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260812 02:36:59.657331 16899 raft_consensus.cc:482] T 88e1b260fa1740b8b9238b91f65053f3 P 6c404a3267bc4a11a6850d91dec7b21d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 02:36:59.657661 16899 raft_consensus.cc:504] T 88e1b260fa1740b8b9238b91f65053f3 P 6c404a3267bc4a11a6850d91dec7b21d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b5778ae04246ec9f152a523a1ef33f" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 36973 } } peers { permanent_uuid: "ed2e8f19bfd642ac95e6b32d054ce8f4" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 34169 } } peers { permanent_uuid: "6c404a3267bc4a11a6850d91dec7b21d" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 35367 } }
I20260812 02:36:59.659358 16899 leader_election.cc:288] T 88e1b260fa1740b8b9238b91f65053f3 P 6c404a3267bc4a11a6850d91dec7b21d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 07b5778ae04246ec9f152a523a1ef33f (127.7.132.130:36973), ed2e8f19bfd642ac95e6b32d054ce8f4 (127.7.132.129:34169)
I20260812 02:36:59.665524 16890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:59.659819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:37199 (local address 127.7.132.130:36973)
0812 02:36:59.660072 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:59.660079 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:59.660105 (+ 26us) server_negotiation.cc:407] Connection header received
0812 02:36:59.660284 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:59.660294 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:59.660399 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:59.660507 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:36:59.661277 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:59.662044 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:59.663809 (+ 1765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:59.664574 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:59.664754 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:59.664835 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:59.665123 (+ 288us) server_negotiation.cc:299] Negotiation successful
0812 02:36:59.665254 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260812 02:36:59.665596 16901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:59.659694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.130:36973 (local address 127.7.132.131:37199)
0812 02:36:59.659893 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:59.659925 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:59.660102 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:59.660541 (+ 439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:59.660546 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:59.660585 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:36:59.660879 (+ 294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:59.660890 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:59.662225 (+ 1335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:59.662231 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:59.663646 (+ 1415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:59.663659 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:59.664769 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:59.664775 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:59.664952 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:59.664981 (+ 29us) client_negotiation.cc:769] Sending connection context
0812 02:36:59.665095 (+ 114us) client_negotiation.cc:240] Negotiation successful
0812 02:36:59.665243 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64,"mutex_wait_us":22}
I20260812 02:36:59.665793 16893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:59.660204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:33901 (local address 127.7.132.129:34169)
0812 02:36:59.660412 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:59.660420 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:59.660845 (+ 425us) server_negotiation.cc:407] Connection header received
0812 02:36:59.661009 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:59.661013 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:59.661074 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:59.661153 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:36:59.661606 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:59.662330 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:59.664042 (+ 1712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:59.664833 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:59.665022 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:59.665094 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:59.665383 (+ 289us) server_negotiation.cc:299] Negotiation successful
0812 02:36:59.665516 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 02:36:59.666319 16906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:59.660031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:34169 (local address 127.7.132.131:33901)
0812 02:36:59.660719 (+ 688us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:59.660737 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 02:36:59.660882 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:36:59.661195 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:36:59.661199 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:36:59.661220 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:36:59.661449 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:59.661455 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:59.662500 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:59.662506 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:59.663879 (+ 1373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:36:59.663890 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:36:59.665036 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:36:59.665042 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:36:59.665210 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:59.665234 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 02:36:59.665340 (+ 106us) client_negotiation.cc:240] Negotiation successful
0812 02:36:59.665464 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":555,"mutex_wait_us":20,"thread_start_us":402,"threads_started":1}
I20260812 02:36:59.666329 16551 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88e1b260fa1740b8b9238b91f65053f3" candidate_uuid: "6c404a3267bc4a11a6850d91dec7b21d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b5778ae04246ec9f152a523a1ef33f" is_pre_election: true
I20260812 02:36:59.666594 16551 raft_consensus.cc:2454] T 88e1b260fa1740b8b9238b91f65053f3 P 07b5778ae04246ec9f152a523a1ef33f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c404a3267bc4a11a6850d91dec7b21d in term 0.
I20260812 02:36:59.667220 16645 leader_election.cc:302] T 88e1b260fa1740b8b9238b91f65053f3 P 6c404a3267bc4a11a6850d91dec7b21d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07b5778ae04246ec9f152a523a1ef33f, 6c404a3267bc4a11a6850d91dec7b21d; no voters:
I20260812 02:36:59.667312 16379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88e1b260fa1740b8b9238b91f65053f3" candidate_uuid: "6c404a3267bc4a11a6850d91dec7b21d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed2e8f19bfd642ac95e6b32d054ce8f4" is_pre_election: true
I20260812 02:36:59.667683 16899 raft_consensus.cc:2781] T 88e1b260fa1740b8b9238b91f65053f3 P 6c404a3267bc4a11a6850d91dec7b21d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 02:36:59.667815 16899 raft_consensus.cc:482] T 88e1b260fa1740b8b9238b91f65053f3 P 6c404a3267bc4a11a6850d91dec7b21d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 02:36:59.667773 16379 raft_consensus.cc:2454] T 88e1b260fa1740b8b9238b91f65053f3 P ed2e8f19bfd642ac95e6b32d054ce8f4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c404a3267bc4a11a6850d91dec7b21d in term 0.
I20260812 02:36:59.667927 16899 raft_consensus.cc:3037] T 88e1b260fa1740b8b9238b91f65053f3 P 6c404a3267bc4a11a6850d91dec7b21d [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:36:59.671272 16899 raft_consensus.cc:504] T 88e1b260fa1740b8b9238b91f65053f3 P 6c404a3267bc4a11a6850d91dec7b21d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b5778ae04246ec9f152a523a1ef33f" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 36973 } } peers { permanent_uuid: "ed2e8f19bfd642ac95e6b32d054ce8f4" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 34169 } } peers { permanent_uuid: "6c404a3267bc4a11a6850d91dec7b21d" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 35367 } }
I20260812 02:36:59.671887 16899 leader_election.cc:288] T 88e1b260fa1740b8b9238b91f65053f3 P 6c404a3267bc4a11a6850d91dec7b21d [CANDIDATE]: Term 1 election: Requested vote from peers 07b5778ae04246ec9f152a523a1ef33f (127.7.132.130:36973), ed2e8f19bfd642ac95e6b32d054ce8f4 (127.7.132.129:34169)
I20260812 02:36:59.672498 16379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88e1b260fa1740b8b9238b91f65053f3" candidate_uuid: "6c404a3267bc4a11a6850d91dec7b21d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed2e8f19bfd642ac95e6b32d054ce8f4"
I20260812 02:36:59.672497 16551 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88e1b260fa1740b8b9238b91f65053f3" candidate_uuid: "6c404a3267bc4a11a6850d91dec7b21d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b5778ae04246ec9f152a523a1ef33f"
I20260812 02:36:59.672749 16379 raft_consensus.cc:3037] T 88e1b260fa1740b8b9238b91f65053f3 P ed2e8f19bfd642ac95e6b32d054ce8f4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:36:59.672749 16551 raft_consensus.cc:3037] T 88e1b260fa1740b8b9238b91f65053f3 P 07b5778ae04246ec9f152a523a1ef33f [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:36:59.675941 16551 raft_consensus.cc:2454] T 88e1b260fa1740b8b9238b91f65053f3 P 07b5778ae04246ec9f152a523a1ef33f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c404a3267bc4a11a6850d91dec7b21d in term 1.
I20260812 02:36:59.675941 16379 raft_consensus.cc:2454] T 88e1b260fa1740b8b9238b91f65053f3 P ed2e8f19bfd642ac95e6b32d054ce8f4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c404a3267bc4a11a6850d91dec7b21d in term 1.
I20260812 02:36:59.676385 16645 leader_election.cc:302] T 88e1b260fa1740b8b9238b91f65053f3 P 6c404a3267bc4a11a6850d91dec7b21d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c404a3267bc4a11a6850d91dec7b21d, ed2e8f19bfd642ac95e6b32d054ce8f4; no voters:
I20260812 02:36:59.676712 16899 raft_consensus.cc:2781] T 88e1b260fa1740b8b9238b91f65053f3 P 6c404a3267bc4a11a6850d91dec7b21d [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:36:59.677187 16899 raft_consensus.cc:686] T 88e1b260fa1740b8b9238b91f65053f3 P 6c404a3267bc4a11a6850d91dec7b21d [term 1 LEADER]: Becoming Leader. State: Replica: 6c404a3267bc4a11a6850d91dec7b21d, State: Running, Role: LEADER
I20260812 02:36:59.677618 16899 consensus_queue.cc:237] T 88e1b260fa1740b8b9238b91f65053f3 P 6c404a3267bc4a11a6850d91dec7b21d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b5778ae04246ec9f152a523a1ef33f" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 36973 } } peers { permanent_uuid: "ed2e8f19bfd642ac95e6b32d054ce8f4" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 34169 } } peers { permanent_uuid: "6c404a3267bc4a11a6850d91dec7b21d" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 35367 } }
I20260812 02:36:59.678388 16906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:59.677862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:33723 (local address 127.7.132.131:37331)
0812 02:36:59.678131 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:59.678254 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:33723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101}
I20260812 02:36:59.681407 15983 catalog_manager.cc:5495] T 88e1b260fa1740b8b9238b91f65053f3 P 6c404a3267bc4a11a6850d91dec7b21d reported cstate change: term changed from 0 to 1, leader changed from <none> to 6c404a3267bc4a11a6850d91dec7b21d (127.7.132.131). New cstate: current_term: 1 leader_uuid: "6c404a3267bc4a11a6850d91dec7b21d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b5778ae04246ec9f152a523a1ef33f" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 36973 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ed2e8f19bfd642ac95e6b32d054ce8f4" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 34169 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c404a3267bc4a11a6850d91dec7b21d" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 35367 } health_report { overall_health: HEALTHY } } }
I20260812 02:36:59.700399 16891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:59.694258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37488 (local address 127.7.132.131:35367)
0812 02:36:59.694423 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:59.694428 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:59.694508 (+ 80us) server_negotiation.cc:407] Connection header received
0812 02:36:59.694693 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:59.694698 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:59.694805 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:59.694956 (+ 151us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 02:36:59.695325 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:59.696437 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:59.697867 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:59.698571 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:59.698688 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:59.699085 (+ 397us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 02:36:59.699822 (+ 737us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 02:36:59.699968 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:59.700034 (+ 66us) server_negotiation.cc:299] Negotiation successful
0812 02:36:59.700121 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 02:36:59.705005 16551 raft_consensus.cc:1261] T 88e1b260fa1740b8b9238b91f65053f3 P 07b5778ae04246ec9f152a523a1ef33f [term 1 FOLLOWER]: Refusing update from remote peer 6c404a3267bc4a11a6850d91dec7b21d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:36:59.705313 16379 raft_consensus.cc:1261] T 88e1b260fa1740b8b9238b91f65053f3 P ed2e8f19bfd642ac95e6b32d054ce8f4 [term 1 FOLLOWER]: Refusing update from remote peer 6c404a3267bc4a11a6850d91dec7b21d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:36:59.705861 16908 consensus_queue.cc:1035] T 88e1b260fa1740b8b9238b91f65053f3 P 6c404a3267bc4a11a6850d91dec7b21d [LEADER]: Connected to new peer: Peer: permanent_uuid: "07b5778ae04246ec9f152a523a1ef33f" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 36973 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:36:59.706358 16899 consensus_queue.cc:1035] T 88e1b260fa1740b8b9238b91f65053f3 P 6c404a3267bc4a11a6850d91dec7b21d [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed2e8f19bfd642ac95e6b32d054ce8f4" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 34169 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:36:59.708973 16903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:59.708421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:33723 (local address 127.7.132.130:34495)
0812 02:36:59.708621 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:59.708752 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:33723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260812 02:36:59.710062 16905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:59.709674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:33723 (local address 127.7.132.129:44321)
0812 02:36:59.709832 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:59.709927 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:33723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260812 02:36:59.714840 16913 mvcc.cc:205] Tried to move back new op lower bound from 7317513091903410176 to 7317513091804090368. Current Snapshot: MvccSnapshot[applied={T|T < 7317513091903410176}]
I20260812 02:36:59.716610 16903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:59.716083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:33723 (local address 127.7.132.130:48959)
0812 02:36:59.716305 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:59.716392 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:33723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123}
I20260812 02:36:59.727249 15983 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41052:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 02:36:59.727620 15983 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41052:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 02:36:59.727975 16906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:59.727212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:33723 (local address 127.7.132.131:44697)
0812 02:36:59.727576 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:59.727710 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:33723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212}
I20260812 02:36:59.735978 15983 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 351974fae6dd4c168a844a01b53d9927: Sending DeleteTablet for 3 replicas of tablet 88e1b260fa1740b8b9238b91f65053f3
I20260812 02:36:59.737082 16700 tablet_service.cc:1514] Processing DeleteTablet for tablet 88e1b260fa1740b8b9238b91f65053f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:36:59 UTC) from {username='oryx', principal='oryx/127.7.132.189@KRBTEST.COM'} at 127.0.0.1:37484
I20260812 02:36:59.737140 16699 tablet_service.cc:1514] Processing DeleteTablet for tablet 88e1b260fa1740b8b9238b91f65053f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:36:59 UTC) from {username='oryx', principal='oryx/127.7.132.189@KRBTEST.COM'} at 127.0.0.1:37484
I20260812 02:36:59.737515 16530 tablet_service.cc:1514] Processing DeleteTablet for tablet 88e1b260fa1740b8b9238b91f65053f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:36:59 UTC) from {username='oryx', principal='oryx/127.7.132.189@KRBTEST.COM'} at 127.0.0.1:58166
I20260812 02:36:59.737789 16926 tablet_replica.cc:330] T 88e1b260fa1740b8b9238b91f65053f3 P 6c404a3267bc4a11a6850d91dec7b21d: stopping tablet replica
I20260812 02:36:59.737905 16353 tablet_service.cc:1514] Processing DeleteTablet for tablet 88e1b260fa1740b8b9238b91f65053f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:36:59 UTC) from {username='oryx', principal='oryx/127.7.132.189@KRBTEST.COM'} at 127.0.0.1:55882
I20260812 02:36:59.737978 16926 raft_consensus.cc:2229] T 88e1b260fa1740b8b9238b91f65053f3 P 6c404a3267bc4a11a6850d91dec7b21d [term 1 LEADER]: Raft consensus shutting down.
I20260812 02:36:59.738590 16926 raft_consensus.cc:2258] T 88e1b260fa1740b8b9238b91f65053f3 P 6c404a3267bc4a11a6850d91dec7b21d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:36:59.739362 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 16179
I20260812 02:36:59.744976 16926 ts_tablet_manager.cc:1905] T 88e1b260fa1740b8b9238b91f65053f3 P 6c404a3267bc4a11a6850d91dec7b21d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:36:59.755373 16928 tablet_replica.cc:330] T 88e1b260fa1740b8b9238b91f65053f3 P ed2e8f19bfd642ac95e6b32d054ce8f4: stopping tablet replica
I20260812 02:36:59.764425 16926 ts_tablet_manager.cc:1918] T 88e1b260fa1740b8b9238b91f65053f3 P 6c404a3267bc4a11a6850d91dec7b21d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 02:36:59.764602 16926 log.cc:1196] T 88e1b260fa1740b8b9238b91f65053f3 P 6c404a3267bc4a11a6850d91dec7b21d: Deleting WAL directory at /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/ts-2/wal/wals/88e1b260fa1740b8b9238b91f65053f3
I20260812 02:36:59.765141 16926 ts_tablet_manager.cc:1939] T 88e1b260fa1740b8b9238b91f65053f3 P 6c404a3267bc4a11a6850d91dec7b21d: Deleting consensus metadata
I20260812 02:36:59.766443 16928 raft_consensus.cc:2229] T 88e1b260fa1740b8b9238b91f65053f3 P ed2e8f19bfd642ac95e6b32d054ce8f4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 02:36:59.767030 16927 tablet_replica.cc:330] T 88e1b260fa1740b8b9238b91f65053f3 P 07b5778ae04246ec9f152a523a1ef33f: stopping tablet replica
I20260812 02:36:59.767225 16927 raft_consensus.cc:2229] T 88e1b260fa1740b8b9238b91f65053f3 P 07b5778ae04246ec9f152a523a1ef33f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 02:36:59.767398 16928 pending_rounds.cc:62] T 88e1b260fa1740b8b9238b91f65053f3 P ed2e8f19bfd642ac95e6b32d054ce8f4: Trying to abort 2 pending ops.
I20260812 02:36:59.767572 16927 raft_consensus.cc:2258] T 88e1b260fa1740b8b9238b91f65053f3 P 07b5778ae04246ec9f152a523a1ef33f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:36:59.767607 16928 pending_rounds.cc:68] T 88e1b260fa1740b8b9238b91f65053f3 P ed2e8f19bfd642ac95e6b32d054ce8f4: Aborting op as it isn't in flight: id { term: 1 index: 2 } timestamp: 7317513091903410176 op_type: WRITE_OP write_request { tablet_id: "88e1b260fa1740b8b9238b91f65053f3" schema { columns { name: "key" type: INT32 is_key: true is_nullable: false immutable: false } columns { name: "val" type: INT32 is_key: false is_nullable: true immutable: false } } row_operations { rows: "<redacted>""\001\003\00090\000\0001\324\000\000" indirect_data: "<redacted>""" } external_consistency_mode: CLIENT_PROPAGATED authz_token { token_data: "\010\367\272\357\323\006\"8\n\ntest-admin\022*\n ea6d5dbc4022425f9f352b57572c88a0\020\001\030\001 \001(\001" signature: "<redacted>""\273?\\\250\247\352\321\014\025\211\301_\374c\237\234na\235D\367\223.\316\344i# Ud\254)\343\325[#\325\363\301\032\'\371;,\235.c\366\356\177{\222\3765R\305\315m[\355\310R\231K" signing_key_seq_num: 0 } } request_id { client_id: "0fc0988521e14556b638819ba10f7e56" seq_no: 0 first_incomplete_seq_no: 0 attempt_no: 0 }
I20260812 02:36:59.768782 16928 pending_rounds.cc:68] T 88e1b260fa1740b8b9238b91f65053f3 P ed2e8f19bfd642ac95e6b32d054ce8f4: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7317513091804090368 op_type: NO_OP noop_request { }
I20260812 02:36:59.768945 16928 raft_consensus.cc:2866] T 88e1b260fa1740b8b9238b91f65053f3 P ed2e8f19bfd642ac95e6b32d054ce8f4 [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260812 02:36:59.769112 16928 raft_consensus.cc:2258] T 88e1b260fa1740b8b9238b91f65053f3 P ed2e8f19bfd642ac95e6b32d054ce8f4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:36:59.769788 16927 ts_tablet_manager.cc:1905] T 88e1b260fa1740b8b9238b91f65053f3 P 07b5778ae04246ec9f152a523a1ef33f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:36:59.770920 16928 ts_tablet_manager.cc:1905] T 88e1b260fa1740b8b9238b91f65053f3 P ed2e8f19bfd642ac95e6b32d054ce8f4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:36:59.776985 16927 ts_tablet_manager.cc:1918] T 88e1b260fa1740b8b9238b91f65053f3 P 07b5778ae04246ec9f152a523a1ef33f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 02:36:59.777169 16927 log.cc:1196] T 88e1b260fa1740b8b9238b91f65053f3 P 07b5778ae04246ec9f152a523a1ef33f: Deleting WAL directory at /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/ts-1/wal/wals/88e1b260fa1740b8b9238b91f65053f3
I20260812 02:36:59.777706 16927 ts_tablet_manager.cc:1939] T 88e1b260fa1740b8b9238b91f65053f3 P 07b5778ae04246ec9f152a523a1ef33f: Deleting consensus metadata
I20260812 02:36:59.778577 15965 catalog_manager.cc:4841] TS 6c404a3267bc4a11a6850d91dec7b21d (127.7.132.131:35367): tablet 88e1b260fa1740b8b9238b91f65053f3 (table test-table [id=ea6d5dbc4022425f9f352b57572c88a0]) successfully deleted
I20260812 02:36:59.778970 16928 ts_tablet_manager.cc:1918] T 88e1b260fa1740b8b9238b91f65053f3 P ed2e8f19bfd642ac95e6b32d054ce8f4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 02:36:59.779124 16928 log.cc:1196] T 88e1b260fa1740b8b9238b91f65053f3 P ed2e8f19bfd642ac95e6b32d054ce8f4: Deleting WAL directory at /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786502156971023-7698-0/minicluster-data/ts-0/wal/wals/88e1b260fa1740b8b9238b91f65053f3
I20260812 02:36:59.779726 16928 ts_tablet_manager.cc:1939] T 88e1b260fa1740b8b9238b91f65053f3 P ed2e8f19bfd642ac95e6b32d054ce8f4: Deleting consensus metadata
W20260812 02:36:59.779932 15965 catalog_manager.cc:4808] TS 6c404a3267bc4a11a6850d91dec7b21d (127.7.132.131:35367): delete failed for tablet 88e1b260fa1740b8b9238b91f65053f3 because the tablet was not found. No further retry: Not found: Tablet not found: 88e1b260fa1740b8b9238b91f65053f3
I20260812 02:36:59.780371 15968 catalog_manager.cc:4841] TS 07b5778ae04246ec9f152a523a1ef33f (127.7.132.130:36973): tablet 88e1b260fa1740b8b9238b91f65053f3 (table test-table [id=ea6d5dbc4022425f9f352b57572c88a0]) successfully deleted
I20260812 02:36:59.785790 15968 catalog_manager.cc:4841] TS ed2e8f19bfd642ac95e6b32d054ce8f4 (127.7.132.129:34169): tablet 88e1b260fa1740b8b9238b91f65053f3 (table test-table [id=ea6d5dbc4022425f9f352b57572c88a0]) successfully deleted
I20260812 02:36:59.787051 16893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:59.740440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55884 (local address 127.7.132.129:34169)
0812 02:36:59.740612 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0812 02:36:59.740618 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:36:59.768350 (+ 27732us) server_negotiation.cc:407] Connection header received
0812 02:36:59.768573 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:36:59.768578 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:36:59.768662 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:36:59.768773 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:36:59.771174 (+ 2401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:59.772324 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:59.774253 (+ 1929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:36:59.774950 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:36:59.775065 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:36:59.779071 (+ 4006us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:36:59.779095 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:36:59.779121 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:36:59.779199 (+ 78us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:36:59.783157 (+ 3958us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:59.784161 (+ 1004us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:59.784171 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:59.784173 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:59.784368 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:36:59.784938 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:36:59.784963 (+ 25us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:36:59.784967 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:36:59.785475 (+ 508us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:36:59.786284 (+ 809us) server_negotiation.cc:1035] Waiting for connection context
0812 02:36:59.786483 (+ 199us) server_negotiation.cc:299] Negotiation successful
0812 02:36:59.786742 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 02:36:59.787739 16427 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:36:59.827752 16870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:59.827293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:33723 (local address 127.0.0.1:34174)
0812 02:36:59.827497 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:59.827589 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:33723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260812 02:36:59.829119 16870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:59.828638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:33723 (local address 127.0.0.1:34178)
0812 02:36:59.828876 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0812 02:36:59.828988 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:33723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150}
I20260812 02:36:59.962639 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 16179
I20260812 02:36:59.999099 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 16449
I20260812 02:37:00.015740 16890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:36:59.999982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58172 (local address 127.7.132.130:36973)
0812 02:37:00.000289 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:00.000298 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:00.000326 (+ 28us) server_negotiation.cc:407] Connection header received
0812 02:37:00.000480 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:00.000484 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:00.000556 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:00.000651 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:37:00.002644 (+ 1993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:00.003550 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:00.005434 (+ 1884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:00.005859 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:00.005962 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:00.009468 (+ 3506us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:37:00.009488 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:37:00.009505 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:37:00.009554 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:37:00.012630 (+ 3076us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:37:00.013553 (+ 923us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:37:00.013561 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:37:00.013564 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:37:00.013708 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:37:00.014180 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:37:00.014184 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:37:00.014186 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:37:00.014569 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:37:00.014800 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0812 02:37:00.015223 (+ 423us) server_negotiation.cc:299] Negotiation successful
0812 02:37:00.015416 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260812 02:37:00.016445 16600 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:37:00.114285 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 16449
I20260812 02:37:00.145462 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 16622
I20260812 02:37:00.161063 16891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:00.146339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37498 (local address 127.7.132.131:35367)
0812 02:37:00.146541 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:00.146547 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:00.146658 (+ 111us) server_negotiation.cc:407] Connection header received
0812 02:37:00.146876 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:00.146892 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:00.146985 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:00.147101 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:37:00.149084 (+ 1983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:00.149940 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:00.151374 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:00.151842 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:00.151990 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:00.154741 (+ 2751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:37:00.154755 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:37:00.154772 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:37:00.154813 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:37:00.158039 (+ 3226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:37:00.158776 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:37:00.158792 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:37:00.158794 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:37:00.158959 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:37:00.159377 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:37:00.159384 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:37:00.159392 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:37:00.159783 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:37:00.160149 (+ 366us) server_negotiation.cc:1035] Waiting for connection context
0812 02:37:00.160439 (+ 290us) server_negotiation.cc:299] Negotiation successful
0812 02:37:00.160719 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260812 02:37:00.161816 16771 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:37:00.298796 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 16622
I20260812 02:37:00.328406 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 15808
I20260812 02:37:00.329697 16019 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:37:00.457499 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 15808
I20260812 02:37:00.487713 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 15940
I20260812 02:37:00.488895 16253 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:37:00.607475 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 15940
2026-08-12T02:37:00Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (10314 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502156971023-7698-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 02:37:00 dist-test-slave-mn4r krb5kdc[16940](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 02:37:00 dist-test-slave-mn4r krb5kdc[16940](info): set up 2 sockets
Aug 12 02:37:00 dist-test-slave-mn4r krb5kdc[16940](info): commencing operation
krb5kdc: starting...
W20260812 02:37:01.822751 7698 mini_kdc.cc:121] Time spent starting KDC: real 1.163s user 0.005s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502156971023-7698-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502156971023-7698-0/krb5kdc/test-user.keytab.
Aug 12 02:37:01 dist-test-slave-mn4r krb5kdc[16940](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502221, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T02:37:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T02:37:02Z Disabled control of system clock
WARNING: no policy specified for oryx/127.7.132.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.132.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.132.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502156971023-7698-0/krb5kdc/oryx_127.7.132.190.keytab.
Entry for principal oryx/127.7.132.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502156971023-7698-0/krb5kdc/oryx_127.7.132.190.keytab.
I20260812 02:37:02.101871 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502156971023-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502156971023-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502156971023-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:38167
--webserver_interface=127.7.132.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502156971023-7698-0/krb5kdc/oryx_127.7.132.190.keytab
--principal=oryx/127.7.132.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:41255
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.132.190:38167
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:37:02.455607 16956 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:37:02.455919 16956 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:37:02.456104 16956 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:37:02.463601 16956 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:37:02.463713 16956 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260812 02:37:02.463781 16956 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:37:02.463877 16956 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:37:02.463968 16956 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:37:02.464080 16956 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:37:02.474521 16956 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:41255
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502156971023-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502156971023-7698-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.132.190:38167
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502156971023-7698-0/krb5kdc/oryx_127.7.132.190.keytab
--principal=oryx/127.7.132.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:38167
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.132.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502156971023-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:37:02.475914 16956 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:37:02.477299 16956 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:37:02.489795 16961 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:37:02.489845 16964 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:37:02.490173 16962 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:37:02.491258 16956 server_base.cc:1034] running on GCE node
I20260812 02:37:02.492242 16956 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:37:02.493904 16956 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:37:02.495177 16956 hybrid_clock.cc:648] HybridClock initialized: now 1786502222495093 us; error 51 us; skew 500 ppm
I20260812 02:37:02.495601 16956 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:37:02 dist-test-slave-mn4r krb5kdc[16940](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502222, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:37:02.499275 16956 init.cc:377] Logged in from keytab as oryx/127.7.132.190@KRBTEST.COM (short username oryx)
I20260812 02:37:02.501060 16956 webserver.cc:460] Webserver started at http://127.7.132.190:43551/ using document root <none> and password file <none>
I20260812 02:37:02.501865 16956 fs_manager.cc:359] Metadata directory not provided
I20260812 02:37:02.502019 16956 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:37:02.502408 16956 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:37:02.505640 16956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502156971023-7698-0/minicluster-data/master-0/data/instance:
uuid: "1c4324eb32464053b83080be530c21b3"
format_stamp: "Formatted at 2026-08-12 02:37:02 on dist-test-slave-mn4r"
server_key: "57aab410634a07e377d434d4ac96492b"
server_key_iv: "c16be107a2d786953cdd62f7aa00a3ee"
server_key_version: "encryptionkey@0"
I20260812 02:37:02.506559 16956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502156971023-7698-0/minicluster-data/master-0/wal/instance:
uuid: "1c4324eb32464053b83080be530c21b3"
format_stamp: "Formatted at 2026-08-12 02:37:02 on dist-test-slave-mn4r"
server_key: "57aab410634a07e377d434d4ac96492b"
server_key_iv: "c16be107a2d786953cdd62f7aa00a3ee"
server_key_version: "encryptionkey@0"
I20260812 02:37:02.512435 16956 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260812 02:37:02.516232 16974 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:37:02.517656 16956 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 02:37:02.517865 16956 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502156971023-7698-0/minicluster-data/master-0/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502156971023-7698-0/minicluster-data/master-0/wal
uuid: "1c4324eb32464053b83080be530c21b3"
format_stamp: "Formatted at 2026-08-12 02:37:02 on dist-test-slave-mn4r"
server_key: "57aab410634a07e377d434d4ac96492b"
server_key_iv: "c16be107a2d786953cdd62f7aa00a3ee"
server_key_version: "encryptionkey@0"
I20260812 02:37:02.518163 16956 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502156971023-7698-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502156971023-7698-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502156971023-7698-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:37:02.537050 16956 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:37:02.541872 16956 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:37:02.542246 16956 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:37:02.564960 16956 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.190:38167
I20260812 02:37:02.564980 17049 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.190:38167 every 8 connection(s)
I20260812 02:37:02.566540 16956 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
I20260812 02:37:02.569890 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 16956
I20260812 02:37:02.570144 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502156971023-7698-0/minicluster-data/master-0/wal/instance
I20260812 02:37:02.570593 17050 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:37:02.570853 7698 external_mini_cluster.cc:1428] Setting key 7d809e3a49602dc95dfe1efe86bc6301
I20260812 02:37:02.586545 17050 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c4324eb32464053b83080be530c21b3: Bootstrap starting.
Aug 12 02:37:02 dist-test-slave-mn4r krb5kdc[16940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502221, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM
I20260812 02:37:02.590764 17050 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1c4324eb32464053b83080be530c21b3: Neither blocks nor log segments found. Creating new log.
I20260812 02:37:02.592283 17050 log.cc:824] T 00000000000000000000000000000000 P 1c4324eb32464053b83080be530c21b3: Log is configured to *not* fsync() on all Append() calls
I20260812 02:37:02.596083 17050 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c4324eb32464053b83080be530c21b3: No bootstrap required, opened a new log
I20260812 02:37:02.598011 17053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:02.572994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44828 (local address 127.7.132.190:38167)
0812 02:37:02.573903 (+ 909us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:02.573919 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:02.573972 (+ 53us) server_negotiation.cc:407] Connection header received
0812 02:37:02.575832 (+ 1860us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:02.575863 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:02.576394 (+ 531us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:02.576877 (+ 483us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:37:02.578332 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:02.580373 (+ 2041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:02.581694 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:02.582897 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:02.583103 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:02.588719 (+ 5616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:37:02.588731 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:37:02.588752 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:37:02.588812 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:37:02.592948 (+ 4136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:37:02.593934 (+ 986us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:37:02.593946 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:37:02.593958 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:37:02.594204 (+ 246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:37:02.594634 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:37:02.594639 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:37:02.594642 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:37:02.595388 (+ 746us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:37:02.595710 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0812 02:37:02.596043 (+ 333us) server_negotiation.cc:299] Negotiation successful
0812 02:37:02.596507 (+ 464us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":646,"thread_start_us":376,"threads_started":1}
I20260812 02:37:02.605191 17050 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1c4324eb32464053b83080be530c21b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c4324eb32464053b83080be530c21b3" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 38167 } }
I20260812 02:37:02.605549 17050 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1c4324eb32464053b83080be530c21b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:37:02.605631 17050 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1c4324eb32464053b83080be530c21b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c4324eb32464053b83080be530c21b3, State: Initialized, Role: FOLLOWER
I20260812 02:37:02.606200 17050 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1c4324eb32464053b83080be530c21b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c4324eb32464053b83080be530c21b3" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 38167 } }
I20260812 02:37:02.606415 17050 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1c4324eb32464053b83080be530c21b3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 02:37:02.606513 17050 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1c4324eb32464053b83080be530c21b3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 02:37:02.606647 17050 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1c4324eb32464053b83080be530c21b3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:37:02.608810 17050 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1c4324eb32464053b83080be530c21b3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c4324eb32464053b83080be530c21b3" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 38167 } }
I20260812 02:37:02.609431 17050 leader_election.cc:302] T 00000000000000000000000000000000 P 1c4324eb32464053b83080be530c21b3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1c4324eb32464053b83080be530c21b3; no voters:
I20260812 02:37:02.610199 17050 leader_election.cc:288] T 00000000000000000000000000000000 P 1c4324eb32464053b83080be530c21b3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 02:37:02.610349 17056 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1c4324eb32464053b83080be530c21b3 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:37:02.611073 17056 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1c4324eb32464053b83080be530c21b3 [term 1 LEADER]: Becoming Leader. State: Replica: 1c4324eb32464053b83080be530c21b3, State: Running, Role: LEADER
I20260812 02:37:02.611589 17056 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1c4324eb32464053b83080be530c21b3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c4324eb32464053b83080be530c21b3" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 38167 } }
I20260812 02:37:02.612363 17050 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1c4324eb32464053b83080be530c21b3 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:37:02.614650 17058 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c4324eb32464053b83080be530c21b3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1c4324eb32464053b83080be530c21b3. Latest consensus state: current_term: 1 leader_uuid: "1c4324eb32464053b83080be530c21b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c4324eb32464053b83080be530c21b3" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 38167 } } }
I20260812 02:37:02.614718 17057 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c4324eb32464053b83080be530c21b3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1c4324eb32464053b83080be530c21b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c4324eb32464053b83080be530c21b3" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 38167 } } }
I20260812 02:37:02.615079 17058 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c4324eb32464053b83080be530c21b3 [sys.catalog]: This master's current role is: LEADER
I20260812 02:37:02.615177 17057 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c4324eb32464053b83080be530c21b3 [sys.catalog]: This master's current role is: LEADER
I20260812 02:37:02.617564 17064 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:37:02.624821 17064 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 02:37:02.636629 17064 catalog_manager.cc:1347] Generated new cluster ID: 7f49b15af40d4eb19ff2cbffe8af8fd8
I20260812 02:37:02.636759 17064 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 02:37:02.652436 17064 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 02:37:02.654367 17064 catalog_manager.cc:1504] Loading token signing keys...
I20260812 02:37:02.666803 17064 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1c4324eb32464053b83080be530c21b3: Generated new TSK 0
I20260812 02:37:02.667749 17064 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.7.132.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.132.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.132.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502156971023-7698-0/krb5kdc/oryx_127.7.132.129.keytab.
Entry for principal oryx/127.7.132.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502156971023-7698-0/krb5kdc/oryx_127.7.132.129.keytab.
I20260812 02:37:02.746342 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502156971023-7698-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502156971023-7698-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502156971023-7698-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502156971023-7698-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.129:0
--local_ip_for_outbound_sockets=127.7.132.129
--webserver_interface=127.7.132.129
--webserver_port=0
--tserver_master_addrs=127.7.132.190:38167
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502156971023-7698-0/krb5kdc/oryx_127.7.132.129.keytab
--principal=oryx/127.7.132.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:41255
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:37:03.120811 17080 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:37:03.121158 17080 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:37:03.121320 17080 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:37:03.128989 17080 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:37:03.129155 17080 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:37:03.129309 17080 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 02:37:03.129449 17080 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.129
I20260812 02:37:03.139586 17080 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:41255
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502156971023-7698-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502156971023-7698-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502156971023-7698-0/krb5kdc/oryx_127.7.132.129.keytab
--principal=oryx/127.7.132.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502156971023-7698-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.132.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.132.190:38167
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.129
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502156971023-7698-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:37:03.141052 17080 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:37:03.142505 17080 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:37:03.156306 17085 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:37:03.156255 17086 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:37:03.156280 17089 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:37:03.157132 17080 server_base.cc:1034] running on GCE node
I20260812 02:37:03.157716 17080 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:37:03.158660 17080 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:37:03.159893 17080 hybrid_clock.cc:648] HybridClock initialized: now 1786502223159820 us; error 53 us; skew 500 ppm
I20260812 02:37:03.160188 17080 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:37:03 dist-test-slave-mn4r krb5kdc[16940](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502223, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.132.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:37:03.169588 17080 init.cc:377] Logged in from keytab as oryx/127.7.132.129@KRBTEST.COM (short username oryx)
I20260812 02:37:03.171103 17080 webserver.cc:460] Webserver started at http://127.7.132.129:45103/ using document root <none> and password file <none>
I20260812 02:37:03.171826 17080 fs_manager.cc:359] Metadata directory not provided
I20260812 02:37:03.171959 17080 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:37:03.172289 17080 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:37:03.175312 17080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502156971023-7698-0/minicluster-data/ts-0/data/instance:
uuid: "e97688088d8542f7a7d708d86b197530"
format_stamp: "Formatted at 2026-08-12 02:37:03 on dist-test-slave-mn4r"
server_key: "88b1f1e1a5ceefc72edf1091bc903fc7"
server_key_iv: "280916963d47d9ff0faa949304b89521"
server_key_version: "encryptionkey@0"
I20260812 02:37:03.176121 17080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502156971023-7698-0/minicluster-data/ts-0/wal/instance:
uuid: "e97688088d8542f7a7d708d86b197530"
format_stamp: "Formatted at 2026-08-12 02:37:03 on dist-test-slave-mn4r"
server_key: "88b1f1e1a5ceefc72edf1091bc903fc7"
server_key_iv: "280916963d47d9ff0faa949304b89521"
server_key_version: "encryptionkey@0"
I20260812 02:37:03.181303 17080 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260812 02:37:03.184867 17097 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:37:03.186273 17080 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 02:37:03.186455 17080 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502156971023-7698-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502156971023-7698-0/minicluster-data/ts-0/wal
uuid: "e97688088d8542f7a7d708d86b197530"
format_stamp: "Formatted at 2026-08-12 02:37:03 on dist-test-slave-mn4r"
server_key: "88b1f1e1a5ceefc72edf1091bc903fc7"
server_key_iv: "280916963d47d9ff0faa949304b89521"
server_key_version: "encryptionkey@0"
I20260812 02:37:03.186717 17080 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502156971023-7698-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502156971023-7698-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502156971023-7698-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:37:03.218111 17080 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:37:03.223645 17080 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:37:03.223956 17080 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:37:03.226832 17080 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:37:03.226960 17080 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:37:03.227111 17080 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:37:03.227186 17080 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 02:37:03 dist-test-slave-mn4r krb5kdc[16940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502223, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.132.129@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM
I20260812 02:37:03.254493 17109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:03.227651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:40401 (local address 127.7.132.190:38167)
0812 02:37:03.230478 (+ 2827us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:03.230485 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:03.230511 (+ 26us) server_negotiation.cc:407] Connection header received
0812 02:37:03.231958 (+ 1447us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:03.231964 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:03.232057 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:03.232192 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:37:03.234603 (+ 2411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:03.235676 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:03.238421 (+ 2745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:03.239131 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:03.239309 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:03.245878 (+ 6569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:37:03.245888 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:37:03.245892 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:37:03.245948 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:37:03.250113 (+ 4165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:37:03.251356 (+ 1243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:37:03.251365 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:37:03.251372 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:37:03.251556 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:37:03.252238 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:37:03.252246 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:37:03.252251 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:37:03.252643 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:37:03.252807 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0812 02:37:03.253843 (+ 1036us) server_negotiation.cc:299] Negotiation successful
0812 02:37:03.254108 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2695,"thread_start_us":2586,"threads_started":1}
I20260812 02:37:03.256114 17110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:03.228321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:38167 (local address 127.7.132.129:40401)
0812 02:37:03.229494 (+ 1173us) negotiation.cc:107] Waiting for socket to connect
0812 02:37:03.229549 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0812 02:37:03.231657 (+ 2108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:37:03.232455 (+ 798us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:37:03.232464 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:37:03.233232 (+ 768us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:37:03.234390 (+ 1158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:03.234430 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:03.235848 (+ 1418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:03.235853 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:03.238204 (+ 2351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:03.238244 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:03.239757 (+ 1513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:03.239773 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:03.240149 (+ 376us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:03.241091 (+ 942us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:37:03.241131 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:37:03.245542 (+ 4411us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:37:03.250321 (+ 4779us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:37:03.250330 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:37:03.250353 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:37:03.251165 (+ 812us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:37:03.251727 (+ 562us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:37:03.251732 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:37:03.251736 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:37:03.252056 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:37:03.252795 (+ 739us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:37:03.252804 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:37:03.253323 (+ 519us) client_negotiation.cc:769] Sending connection context
0812 02:37:03.253717 (+ 394us) client_negotiation.cc:240] Negotiation successful
0812 02:37:03.254343 (+ 626us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":887,"thread_start_us":400,"threads_started":1}
I20260812 02:37:03.270714 17080 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.129:42457
I20260812 02:37:03.270802 17237 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.129:42457 every 8 connection(s)
I20260812 02:37:03.272086 17080 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502156971023-7698-0/minicluster-data/ts-0/data/info.pb
I20260812 02:37:03.274330 17238 heartbeater.cc:347] Connected to a master server at 127.7.132.190:38167
I20260812 02:37:03.274590 17238 heartbeater.cc:464] Registering TS with master...
I20260812 02:37:03.275277 17238 heartbeater.cc:511] Master 127.7.132.190:38167 requested a full tablet report, sending...
I20260812 02:37:03.277527 16998 ts_manager.cc:194] Registered new tserver with Master: e97688088d8542f7a7d708d86b197530 (127.7.132.129:42457)
I20260812 02:37:03.278398 17214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:03.263139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:38167 (local address 127.7.132.129:51371)
0812 02:37:03.263986 (+ 847us) negotiation.cc:107] Waiting for socket to connect
0812 02:37:03.264010 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0812 02:37:03.264249 (+ 239us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:37:03.264682 (+ 433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:37:03.264686 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:37:03.265412 (+ 726us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:37:03.266289 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:03.266302 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:03.267742 (+ 1440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:03.267749 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:03.269199 (+ 1450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:03.269212 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:03.270405 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:03.270410 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:03.270651 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:03.271432 (+ 781us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:37:03.271457 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:37:03.272242 (+ 785us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:37:03.275744 (+ 3502us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:37:03.275749 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:37:03.275755 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:37:03.276261 (+ 506us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:37:03.276690 (+ 429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:37:03.276696 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:37:03.276700 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:37:03.276837 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:37:03.277396 (+ 559us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:37:03.277409 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:37:03.277591 (+ 182us) client_negotiation.cc:769] Sending connection context
0812 02:37:03.277733 (+ 142us) client_negotiation.cc:240] Negotiation successful
0812 02:37:03.278012 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":635,"thread_start_us":470,"threads_started":1}
I20260812 02:37:03.278477 17109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:03.263029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:51371 (local address 127.7.132.190:38167)
0812 02:37:03.263289 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:03.263297 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:03.264107 (+ 810us) server_negotiation.cc:407] Connection header received
0812 02:37:03.264424 (+ 317us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:03.264430 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:03.264521 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:03.264644 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:37:03.266538 (+ 1894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:03.267514 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:03.269341 (+ 1827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:03.270206 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:03.270351 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:03.272440 (+ 2089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:37:03.272447 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:37:03.272453 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:37:03.272496 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:37:03.275624 (+ 3128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:37:03.276361 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:37:03.276368 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:37:03.276371 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:37:03.276564 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:37:03.276947 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:37:03.276953 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:37:03.276956 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:37:03.277279 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:37:03.277451 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0812 02:37:03.277849 (+ 398us) server_negotiation.cc:299] Negotiation successful
0812 02:37:03.278140 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260812 02:37:03.279649 16998 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.132.129@KRBTEST.COM'} at 127.7.132.129:40401
I20260812 02:37:03.280329 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 17080
I20260812 02:37:03.280539 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502156971023-7698-0/minicluster-data/ts-0/wal/instance
I20260812 02:37:03.281309 7698 external_mini_cluster.cc:1428] Setting key a29bdbcb8fe4c5ed04f53abb96ba15ed
WARNING: no policy specified for oryx/127.7.132.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.132.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.132.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502156971023-7698-0/krb5kdc/oryx_127.7.132.130.keytab.
Entry for principal oryx/127.7.132.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502156971023-7698-0/krb5kdc/oryx_127.7.132.130.keytab.
I20260812 02:37:03.337090 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502156971023-7698-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502156971023-7698-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502156971023-7698-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502156971023-7698-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.130:0
--local_ip_for_outbound_sockets=127.7.132.130
--webserver_interface=127.7.132.130
--webserver_port=0
--tserver_master_addrs=127.7.132.190:38167
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502156971023-7698-0/krb5kdc/oryx_127.7.132.130.keytab
--principal=oryx/127.7.132.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:41255
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:37:03.680430 17243 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:37:03.680714 17243 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:37:03.680899 17243 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:37:03.688405 17243 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:37:03.688521 17243 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:37:03.688689 17243 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 02:37:03.688813 17243 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.130
I20260812 02:37:03.699057 17243 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:41255
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502156971023-7698-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502156971023-7698-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502156971023-7698-0/krb5kdc/oryx_127.7.132.130.keytab
--principal=oryx/127.7.132.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502156971023-7698-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.132.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.132.190:38167
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.130
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502156971023-7698-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:37:03.700489 17243 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:37:03.701880 17243 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:37:03.715301 17253 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:37:03.715267 17250 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:37:03.715377 17249 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:37:03.716187 17243 server_base.cc:1034] running on GCE node
I20260812 02:37:03.716806 17243 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:37:03.717697 17243 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:37:03.718931 17243 hybrid_clock.cc:648] HybridClock initialized: now 1786502223718871 us; error 39 us; skew 500 ppm
I20260812 02:37:03.719246 17243 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:37:03 dist-test-slave-mn4r krb5kdc[16940](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502223, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.132.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:37:03.722560 17243 init.cc:377] Logged in from keytab as oryx/127.7.132.130@KRBTEST.COM (short username oryx)
I20260812 02:37:03.724020 17243 webserver.cc:460] Webserver started at http://127.7.132.130:36735/ using document root <none> and password file <none>
I20260812 02:37:03.724694 17243 fs_manager.cc:359] Metadata directory not provided
I20260812 02:37:03.724783 17243 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:37:03.725068 17243 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:37:03.728073 17243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502156971023-7698-0/minicluster-data/ts-1/data/instance:
uuid: "a0a4efea0cec47928cd91fe2ec6ce02e"
format_stamp: "Formatted at 2026-08-12 02:37:03 on dist-test-slave-mn4r"
server_key: "39a5e370ec767897205f5bcfdd3f1c4a"
server_key_iv: "c6a07d822622bc5e17b3fcfe599feb47"
server_key_version: "encryptionkey@0"
I20260812 02:37:03.728823 17243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502156971023-7698-0/minicluster-data/ts-1/wal/instance:
uuid: "a0a4efea0cec47928cd91fe2ec6ce02e"
format_stamp: "Formatted at 2026-08-12 02:37:03 on dist-test-slave-mn4r"
server_key: "39a5e370ec767897205f5bcfdd3f1c4a"
server_key_iv: "c6a07d822622bc5e17b3fcfe599feb47"
server_key_version: "encryptionkey@0"
I20260812 02:37:03.733831 17243 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260812 02:37:03.737241 17263 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:37:03.738602 17243 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 02:37:03.738730 17243 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502156971023-7698-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502156971023-7698-0/minicluster-data/ts-1/wal
uuid: "a0a4efea0cec47928cd91fe2ec6ce02e"
format_stamp: "Formatted at 2026-08-12 02:37:03 on dist-test-slave-mn4r"
server_key: "39a5e370ec767897205f5bcfdd3f1c4a"
server_key_iv: "c6a07d822622bc5e17b3fcfe599feb47"
server_key_version: "encryptionkey@0"
I20260812 02:37:03.738915 17243 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502156971023-7698-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502156971023-7698-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502156971023-7698-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:37:03.761266 17243 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:37:03.765089 17243 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:37:03.765316 17243 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:37:03.767990 17243 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:37:03.768100 17243 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:37:03.768289 17243 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:37:03.768371 17243 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 02:37:03 dist-test-slave-mn4r krb5kdc[16940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502223, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.132.130@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM
I20260812 02:37:03.793001 17273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:03.769138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:38167 (local address 127.7.132.130:57967)
0812 02:37:03.769919 (+ 781us) negotiation.cc:107] Waiting for socket to connect
0812 02:37:03.769957 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 02:37:03.771957 (+ 2000us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:37:03.772926 (+ 969us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:37:03.772937 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:37:03.773821 (+ 884us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:37:03.774957 (+ 1136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:03.774998 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:03.776477 (+ 1479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:03.776483 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:03.777786 (+ 1303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:03.777799 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:03.779108 (+ 1309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:03.779122 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:03.779527 (+ 405us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:03.780531 (+ 1004us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:37:03.780571 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:37:03.784367 (+ 3796us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:37:03.788129 (+ 3762us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:37:03.788143 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:37:03.788172 (+ 29us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:37:03.789047 (+ 875us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:37:03.789531 (+ 484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:37:03.789536 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:37:03.789540 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:37:03.789888 (+ 348us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:37:03.790635 (+ 747us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:37:03.790644 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:37:03.791064 (+ 420us) client_negotiation.cc:769] Sending connection context
0812 02:37:03.791417 (+ 353us) client_negotiation.cc:240] Negotiation successful
0812 02:37:03.791859 (+ 442us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":541,"thread_start_us":340,"threads_started":1}
I20260812 02:37:03.793933 17109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:03.768422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:57967 (local address 127.7.132.190:38167)
0812 02:37:03.768617 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:03.768640 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:03.770075 (+ 1435us) server_negotiation.cc:407] Connection header received
0812 02:37:03.772345 (+ 2270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:03.772352 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:03.772473 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:03.772624 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:37:03.775196 (+ 2572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:03.776313 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:03.777940 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:03.778553 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:03.778713 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:03.784717 (+ 6004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:37:03.784726 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:37:03.784731 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:37:03.784781 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:37:03.787960 (+ 3179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:37:03.789230 (+ 1270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:37:03.789239 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:37:03.789243 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:37:03.789409 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:37:03.790120 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:37:03.790132 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:37:03.790134 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:37:03.790492 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:37:03.790666 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0812 02:37:03.793505 (+ 2839us) server_negotiation.cc:299] Negotiation successful
0812 02:37:03.793693 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 02:37:03.817694 17109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:03.801263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:39841 (local address 127.7.132.190:38167)
0812 02:37:03.801502 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:03.801508 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:03.801916 (+ 408us) server_negotiation.cc:407] Connection header received
0812 02:37:03.802229 (+ 313us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:03.802236 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:03.802333 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:03.802443 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:37:03.803953 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:03.804838 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:03.807388 (+ 2550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:03.807973 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:03.808123 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:03.810956 (+ 2833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:37:03.810966 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:37:03.810974 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:37:03.811025 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:37:03.814271 (+ 3246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:37:03.815279 (+ 1008us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:37:03.815288 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:37:03.815294 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:37:03.815467 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:37:03.815856 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:37:03.815861 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:37:03.815865 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:37:03.816358 (+ 493us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:37:03.816546 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0812 02:37:03.817085 (+ 539us) server_negotiation.cc:299] Negotiation successful
0812 02:37:03.817320 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260812 02:37:03.817791 17347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:03.800877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:38167 (local address 127.7.132.130:39841)
0812 02:37:03.801784 (+ 907us) negotiation.cc:107] Waiting for socket to connect
0812 02:37:03.801811 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0812 02:37:03.802001 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:37:03.802517 (+ 516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:37:03.802525 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:37:03.803292 (+ 767us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:37:03.803791 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:03.803809 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:03.804982 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:03.804989 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:03.807158 (+ 2169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:03.807180 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:03.808200 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:03.808211 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:03.808503 (+ 292us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:03.809368 (+ 865us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:37:03.809401 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:37:03.810719 (+ 1318us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:37:03.814409 (+ 3690us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:37:03.814414 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:37:03.814422 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:37:03.815095 (+ 673us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:37:03.815595 (+ 500us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:37:03.815599 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:37:03.815601 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:37:03.815727 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:37:03.816473 (+ 746us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:37:03.816477 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:37:03.816736 (+ 259us) client_negotiation.cc:769] Sending connection context
0812 02:37:03.816970 (+ 234us) client_negotiation.cc:240] Negotiation successful
0812 02:37:03.817333 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":704,"thread_start_us":506,"threads_started":1}
I20260812 02:37:03.828644 17243 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.130:34695
I20260812 02:37:03.828663 17401 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.130:34695 every 8 connection(s)
I20260812 02:37:03.829918 17243 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502156971023-7698-0/minicluster-data/ts-1/data/info.pb
I20260812 02:37:03.832094 17402 heartbeater.cc:347] Connected to a master server at 127.7.132.190:38167
I20260812 02:37:03.832335 17402 heartbeater.cc:464] Registering TS with master...
I20260812 02:37:03.833002 17402 heartbeater.cc:511] Master 127.7.132.190:38167 requested a full tablet report, sending...
I20260812 02:37:03.834569 16998 ts_manager.cc:194] Registered new tserver with Master: a0a4efea0cec47928cd91fe2ec6ce02e (127.7.132.130:34695)
I20260812 02:37:03.835492 16998 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.132.130@KRBTEST.COM'} at 127.7.132.130:57967
I20260812 02:37:03.835541 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 17243
I20260812 02:37:03.835775 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502156971023-7698-0/minicluster-data/ts-1/wal/instance
I20260812 02:37:03.836350 7698 external_mini_cluster.cc:1428] Setting key 138fc95ac65c52bd0a7571e5f7153660
WARNING: no policy specified for oryx/127.7.132.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.132.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.132.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502156971023-7698-0/krb5kdc/oryx_127.7.132.131.keytab.
Entry for principal oryx/127.7.132.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502156971023-7698-0/krb5kdc/oryx_127.7.132.131.keytab.
I20260812 02:37:03.899616 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502156971023-7698-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502156971023-7698-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502156971023-7698-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502156971023-7698-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.131:0
--local_ip_for_outbound_sockets=127.7.132.131
--webserver_interface=127.7.132.131
--webserver_port=0
--tserver_master_addrs=127.7.132.190:38167
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502156971023-7698-0/krb5kdc/oryx_127.7.132.131.keytab
--principal=oryx/127.7.132.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:41255
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:37:04.252903 17407 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:37:04.253201 17407 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:37:04.253367 17407 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:37:04.260950 17407 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:37:04.261070 17407 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:37:04.261245 17407 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 02:37:04.261368 17407 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.131
I20260812 02:37:04.271368 17407 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:41255
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502156971023-7698-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502156971023-7698-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502156971023-7698-0/krb5kdc/oryx_127.7.132.131.keytab
--principal=oryx/127.7.132.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502156971023-7698-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.132.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.132.190:38167
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.131
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502156971023-7698-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:37:04.272620 17407 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:37:04.273999 17407 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 02:37:04.283155 17238 heartbeater.cc:503] Master 127.7.132.190:38167 was elected leader, sending a full tablet report...
W20260812 02:37:04.287380 17412 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:37:04.287403 17414 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:37:04.287804 17416 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:37:04.288419 17407 server_base.cc:1034] running on GCE node
I20260812 02:37:04.289098 17407 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:37:04.290251 17407 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:37:04.291524 17407 hybrid_clock.cc:648] HybridClock initialized: now 1786502224291455 us; error 37 us; skew 500 ppm
I20260812 02:37:04.291874 17407 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:37:04 dist-test-slave-mn4r krb5kdc[16940](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502224, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.132.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:37:04.295625 17407 init.cc:377] Logged in from keytab as oryx/127.7.132.131@KRBTEST.COM (short username oryx)
I20260812 02:37:04.297245 17407 webserver.cc:460] Webserver started at http://127.7.132.131:35679/ using document root <none> and password file <none>
I20260812 02:37:04.298004 17407 fs_manager.cc:359] Metadata directory not provided
I20260812 02:37:04.298175 17407 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:37:04.298527 17407 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:37:04.301630 17407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502156971023-7698-0/minicluster-data/ts-2/data/instance:
uuid: "68b44fe628d849aea0058efe05f83434"
format_stamp: "Formatted at 2026-08-12 02:37:04 on dist-test-slave-mn4r"
server_key: "b2b5587b1295dccef772c0b75ac656a5"
server_key_iv: "34ca65189a1d0f8f109c34bd21d7293a"
server_key_version: "encryptionkey@0"
I20260812 02:37:04.302526 17407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502156971023-7698-0/minicluster-data/ts-2/wal/instance:
uuid: "68b44fe628d849aea0058efe05f83434"
format_stamp: "Formatted at 2026-08-12 02:37:04 on dist-test-slave-mn4r"
server_key: "b2b5587b1295dccef772c0b75ac656a5"
server_key_iv: "34ca65189a1d0f8f109c34bd21d7293a"
server_key_version: "encryptionkey@0"
I20260812 02:37:04.308061 17407 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.002s
I20260812 02:37:04.311892 17424 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:37:04.313390 17407 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 02:37:04.313587 17407 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502156971023-7698-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502156971023-7698-0/minicluster-data/ts-2/wal
uuid: "68b44fe628d849aea0058efe05f83434"
format_stamp: "Formatted at 2026-08-12 02:37:04 on dist-test-slave-mn4r"
server_key: "b2b5587b1295dccef772c0b75ac656a5"
server_key_iv: "34ca65189a1d0f8f109c34bd21d7293a"
server_key_version: "encryptionkey@0"
I20260812 02:37:04.313827 17407 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502156971023-7698-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502156971023-7698-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502156971023-7698-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:37:04.343546 17407 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:37:04.348392 17407 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:37:04.348698 17407 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:37:04.351635 17407 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:37:04.351745 17407 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:37:04.351920 17407 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:37:04.352000 17407 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 02:37:04 dist-test-slave-mn4r krb5kdc[16940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502224, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.132.131@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM
I20260812 02:37:04.376303 17433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:04.352191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:35371 (local address 127.7.132.190:38167)
0812 02:37:04.352926 (+ 735us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:04.352934 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:04.354068 (+ 1134us) server_negotiation.cc:407] Connection header received
0812 02:37:04.356529 (+ 2461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:04.356537 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:04.356627 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:04.356699 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:37:04.359467 (+ 2768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:04.360562 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:04.363119 (+ 2557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:04.363772 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:04.363919 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:04.368850 (+ 4931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:37:04.368860 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:37:04.368867 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:37:04.368916 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:37:04.372169 (+ 3253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:37:04.373029 (+ 860us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:37:04.373033 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:37:04.373035 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:37:04.373188 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:37:04.373621 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:37:04.373624 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:37:04.373627 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:37:04.373904 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:37:04.374062 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0812 02:37:04.375393 (+ 1331us) server_negotiation.cc:299] Negotiation successful
0812 02:37:04.375716 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":606,"thread_start_us":470,"threads_started":1}
I20260812 02:37:04.376855 17434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:04.352914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:38167 (local address 127.7.132.131:35371)
0812 02:37:04.353863 (+ 949us) negotiation.cc:107] Waiting for socket to connect
0812 02:37:04.353925 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0812 02:37:04.356138 (+ 2213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:37:04.357306 (+ 1168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:37:04.357314 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:37:04.358213 (+ 899us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:37:04.359196 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:04.359232 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:04.360720 (+ 1488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:04.360729 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:04.362957 (+ 2228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:04.362969 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:04.364051 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:04.364060 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:04.364277 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:04.365105 (+ 828us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:37:04.365141 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:37:04.368606 (+ 3465us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:37:04.372312 (+ 3706us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:37:04.372322 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:37:04.372338 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:37:04.372854 (+ 516us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:37:04.373305 (+ 451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:37:04.373310 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:37:04.373314 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:37:04.373496 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:37:04.374043 (+ 547us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:37:04.374051 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:37:04.374487 (+ 436us) client_negotiation.cc:769] Sending connection context
0812 02:37:04.374883 (+ 396us) client_negotiation.cc:240] Negotiation successful
0812 02:37:04.375547 (+ 664us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":635,"thread_start_us":398,"threads_started":1}
I20260812 02:37:04.401093 17433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:04.383623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:35753 (local address 127.7.132.190:38167)
0812 02:37:04.383860 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:04.383870 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:04.384642 (+ 772us) server_negotiation.cc:407] Connection header received
0812 02:37:04.384856 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:04.384865 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:04.384979 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:04.385097 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:37:04.386616 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:04.387309 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:04.390045 (+ 2736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:04.390627 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:04.390764 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:04.393835 (+ 3071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:37:04.393841 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:37:04.393844 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:37:04.393876 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:37:04.397564 (+ 3688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:37:04.398705 (+ 1141us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:37:04.398710 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:37:04.398713 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:37:04.398859 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:37:04.399445 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:37:04.399452 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:37:04.399457 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:37:04.399869 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:37:04.400047 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0812 02:37:04.400486 (+ 439us) server_negotiation.cc:299] Negotiation successful
0812 02:37:04.400736 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260812 02:37:04.401244 17507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:04.383671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:38167 (local address 127.7.132.131:35753)
0812 02:37:04.384521 (+ 850us) negotiation.cc:107] Waiting for socket to connect
0812 02:37:04.384540 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 02:37:04.384717 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:37:04.385163 (+ 446us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:37:04.385171 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:37:04.385918 (+ 747us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:37:04.386424 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:04.386441 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:04.387482 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:04.387491 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:04.389814 (+ 2323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:04.389837 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:04.390823 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:04.390831 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:04.391145 (+ 314us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:04.391993 (+ 848us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:37:04.392026 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:37:04.393616 (+ 1590us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:37:04.397721 (+ 4105us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:37:04.397726 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:37:04.397730 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:37:04.398541 (+ 811us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:37:04.399061 (+ 520us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:37:04.399067 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:37:04.399071 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:37:04.399228 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:37:04.400011 (+ 783us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:37:04.400016 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:37:04.400216 (+ 200us) client_negotiation.cc:769] Sending connection context
0812 02:37:04.400364 (+ 148us) client_negotiation.cc:240] Negotiation successful
0812 02:37:04.400719 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":583,"thread_start_us":400,"threads_started":1}
I20260812 02:37:04.409592 17407 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.131:33969
I20260812 02:37:04.409638 17563 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.131:33969 every 8 connection(s)
I20260812 02:37:04.410984 17407 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502156971023-7698-0/minicluster-data/ts-2/data/info.pb
I20260812 02:37:04.413280 17564 heartbeater.cc:347] Connected to a master server at 127.7.132.190:38167
I20260812 02:37:04.413565 17564 heartbeater.cc:464] Registering TS with master...
I20260812 02:37:04.414419 17564 heartbeater.cc:511] Master 127.7.132.190:38167 requested a full tablet report, sending...
I20260812 02:37:04.415992 16998 ts_manager.cc:194] Registered new tserver with Master: 68b44fe628d849aea0058efe05f83434 (127.7.132.131:33969)
I20260812 02:37:04.416867 16998 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.132.131@KRBTEST.COM'} at 127.7.132.131:35371
I20260812 02:37:04.417865 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 17407
I20260812 02:37:04.418048 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502156971023-7698-0/minicluster-data/ts-2/wal/instance
I20260812 02:37:04.418718 7698 external_mini_cluster.cc:1428] Setting key 989f725138bff6e4dd58ea9d70ec7c8f
I20260812 02:37:04.422212 7698 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 02:37:04 dist-test-slave-mn4r krb5kdc[16940](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502224, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 02:37:04 dist-test-slave-mn4r krb5kdc[16940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 02:37:04 dist-test-slave-mn4r krb5kdc[16940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 02:37:04.838346 17402 heartbeater.cc:503] Master 127.7.132.190:38167 was elected leader, sending a full tablet report...
I20260812 02:37:05.420182 17564 heartbeater.cc:503] Master 127.7.132.190:38167 was elected leader, sending a full tablet report...
W20260812 02:37:05.576411 17572 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.099s user 0.000s sys 0.019s
W20260812 02:37:05.577582 17433 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:37:04.472034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44844 (local address 127.7.132.190:38167)
0812 02:37:04.472283 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:04.472291 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:04.472491 (+ 200us) server_negotiation.cc:407] Connection header received
0812 02:37:04.472743 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:04.472749 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:04.472815 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:04.472924 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:37:04.474341 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:04.475066 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:04.476314 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:04.476745 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:04.476885 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:05.577340 (+1100455us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44844: BlockingRecv error: recv got EOF from 127.0.0.1:44844 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
Aug 12 02:37:05 dist-test-slave-mn4r krb5kdc[16940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502224, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM
I20260812 02:37:05.594816 17433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:05.581411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44856 (local address 127.7.132.190:38167)
0812 02:37:05.581635 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:05.581640 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:05.581983 (+ 343us) server_negotiation.cc:407] Connection header received
0812 02:37:05.582178 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:05.582187 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:05.582294 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:05.582409 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:37:05.583642 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:05.584370 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:05.585578 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:05.585994 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:05.586155 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:05.589856 (+ 3701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:37:05.589861 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:37:05.589864 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:37:05.589897 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:37:05.592550 (+ 2653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:37:05.593166 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:37:05.593170 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:37:05.593172 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:37:05.593287 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:37:05.593642 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:37:05.593645 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:37:05.593647 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:37:05.593891 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:37:05.594161 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0812 02:37:05.594407 (+ 246us) server_negotiation.cc:299] Negotiation successful
0812 02:37:05.594581 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 02:37:05.599694 16998 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44856:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 02:37:05.606678 16998 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 02:37:05.640295 17588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:05.619665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59376 (local address 127.7.132.131:33969)
0812 02:37:05.630416 (+ 10751us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:05.630429 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:05.630470 (+ 41us) server_negotiation.cc:407] Connection header received
0812 02:37:05.630616 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:05.630627 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:05.630953 (+ 326us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:05.632366 (+ 1413us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:37:05.632669 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:05.634678 (+ 2009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:05.637646 (+ 2968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:05.639053 (+ 1407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:05.639200 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:05.639438 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0812 02:37:05.639804 (+ 366us) server_negotiation.cc:299] Negotiation successful
0812 02:37:05.639982 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10527,"thread_start_us":11477,"threads_started":1}
I20260812 02:37:05.640419 17586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:05.618989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.131:33969 (local address 127.0.0.1:59376)
0812 02:37:05.620437 (+ 1448us) negotiation.cc:107] Waiting for socket to connect
0812 02:37:05.620492 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0812 02:37:05.620677 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:37:05.631499 (+ 10822us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:37:05.631511 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:37:05.631556 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:37:05.632107 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:05.632124 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:05.634865 (+ 2741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:05.634874 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:05.637472 (+ 2598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:05.637490 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:05.639248 (+ 1758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:05.639255 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:05.639570 (+ 315us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:05.639613 (+ 43us) client_negotiation.cc:769] Sending connection context
0812 02:37:05.639759 (+ 146us) client_negotiation.cc:240] Negotiation successful
0812 02:37:05.639965 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1185,"mutex_wait_us":96,"thread_start_us":12159,"threads_started":1}
I20260812 02:37:05.643077 17585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:05.619233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.130:34695 (local address 127.0.0.1:44494)
0812 02:37:05.630964 (+ 11731us) negotiation.cc:107] Waiting for socket to connect
0812 02:37:05.630986 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 02:37:05.631107 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:37:05.635787 (+ 4680us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:37:05.635793 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:37:05.635825 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:37:05.636330 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:05.636348 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:05.638051 (+ 1703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:05.638058 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:05.640141 (+ 2083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:05.640155 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:05.642184 (+ 2029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:05.642191 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:05.642366 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:05.642402 (+ 36us) client_negotiation.cc:769] Sending connection context
0812 02:37:05.642553 (+ 151us) client_negotiation.cc:240] Negotiation successful
0812 02:37:05.642683 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11592,"mutex_wait_us":125,"thread_start_us":818,"threads_started":1}
I20260812 02:37:05.643103 17587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:05.619554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44494 (local address 127.7.132.130:34695)
0812 02:37:05.635221 (+ 15667us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:05.635229 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:05.635257 (+ 28us) server_negotiation.cc:407] Connection header received
0812 02:37:05.635378 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:05.635386 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:05.635611 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:05.635816 (+ 205us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:37:05.636535 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:05.637919 (+ 1384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:05.640319 (+ 2400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:05.641819 (+ 1500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:05.641989 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:05.642239 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0812 02:37:05.642610 (+ 371us) server_negotiation.cc:299] Negotiation successful
0812 02:37:05.642778 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15513,"thread_start_us":15375,"threads_started":1}
I20260812 02:37:05.643909 17589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:05.620802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:42457 (local address 127.0.0.1:36814)
0812 02:37:05.634849 (+ 14047us) negotiation.cc:107] Waiting for socket to connect
0812 02:37:05.634871 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 02:37:05.635023 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:37:05.636542 (+ 1519us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:37:05.636547 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:37:05.636570 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:37:05.636959 (+ 389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:05.636968 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:05.638777 (+ 1809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:05.638782 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:05.640968 (+ 2186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:05.640983 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:05.643168 (+ 2185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:05.643177 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:05.643354 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:05.643383 (+ 29us) client_negotiation.cc:769] Sending connection context
0812 02:37:05.643493 (+ 110us) client_negotiation.cc:240] Negotiation successful
0812 02:37:05.643627 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13907,"mutex_wait_us":4,"thread_start_us":14313,"threads_started":1}
I20260812 02:37:05.644073 17590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:05.621144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36814 (local address 127.7.132.129:42457)
0812 02:37:05.634591 (+ 13447us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:05.634603 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:05.636032 (+ 1429us) server_negotiation.cc:407] Connection header received
0812 02:37:05.636140 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:05.636147 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:05.636358 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:05.636553 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:37:05.637135 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:05.638614 (+ 1479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:05.641132 (+ 2518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:05.642670 (+ 1538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:05.642816 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:05.643010 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0812 02:37:05.643611 (+ 601us) server_negotiation.cc:299] Negotiation successful
0812 02:37:05.643778 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13259,"thread_start_us":13097,"threads_started":1}
I20260812 02:37:05.645097 17484 tablet_service.cc:1467] Processing CreateTablet for tablet 8c8abcd1a2874017bbc453addabcf6c6 (DEFAULT_TABLE table=test-table [id=4d448d228ef047719c912472c4f46f85]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:37:05.646510 17484 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c8abcd1a2874017bbc453addabcf6c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:37:05.647472 17156 tablet_service.cc:1467] Processing CreateTablet for tablet 8c8abcd1a2874017bbc453addabcf6c6 (DEFAULT_TABLE table=test-table [id=4d448d228ef047719c912472c4f46f85]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:37:05.648576 17156 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c8abcd1a2874017bbc453addabcf6c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:37:05.649413 17320 tablet_service.cc:1467] Processing CreateTablet for tablet 8c8abcd1a2874017bbc453addabcf6c6 (DEFAULT_TABLE table=test-table [id=4d448d228ef047719c912472c4f46f85]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:37:05.650768 17320 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c8abcd1a2874017bbc453addabcf6c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:37:05.659288 17591 tablet_bootstrap.cc:492] T 8c8abcd1a2874017bbc453addabcf6c6 P e97688088d8542f7a7d708d86b197530: Bootstrap starting.
I20260812 02:37:05.662102 17592 tablet_bootstrap.cc:492] T 8c8abcd1a2874017bbc453addabcf6c6 P 68b44fe628d849aea0058efe05f83434: Bootstrap starting.
I20260812 02:37:05.663681 17591 tablet_bootstrap.cc:654] T 8c8abcd1a2874017bbc453addabcf6c6 P e97688088d8542f7a7d708d86b197530: Neither blocks nor log segments found. Creating new log.
I20260812 02:37:05.665040 17591 log.cc:824] T 8c8abcd1a2874017bbc453addabcf6c6 P e97688088d8542f7a7d708d86b197530: Log is configured to *not* fsync() on all Append() calls
I20260812 02:37:05.665190 17593 tablet_bootstrap.cc:492] T 8c8abcd1a2874017bbc453addabcf6c6 P a0a4efea0cec47928cd91fe2ec6ce02e: Bootstrap starting.
I20260812 02:37:05.665766 17592 tablet_bootstrap.cc:654] T 8c8abcd1a2874017bbc453addabcf6c6 P 68b44fe628d849aea0058efe05f83434: Neither blocks nor log segments found. Creating new log.
I20260812 02:37:05.667024 17592 log.cc:824] T 8c8abcd1a2874017bbc453addabcf6c6 P 68b44fe628d849aea0058efe05f83434: Log is configured to *not* fsync() on all Append() calls
I20260812 02:37:05.668699 17593 tablet_bootstrap.cc:654] T 8c8abcd1a2874017bbc453addabcf6c6 P a0a4efea0cec47928cd91fe2ec6ce02e: Neither blocks nor log segments found. Creating new log.
I20260812 02:37:05.668879 17591 tablet_bootstrap.cc:492] T 8c8abcd1a2874017bbc453addabcf6c6 P e97688088d8542f7a7d708d86b197530: No bootstrap required, opened a new log
I20260812 02:37:05.669163 17591 ts_tablet_manager.cc:1397] T 8c8abcd1a2874017bbc453addabcf6c6 P e97688088d8542f7a7d708d86b197530: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260812 02:37:05.669754 17593 log.cc:824] T 8c8abcd1a2874017bbc453addabcf6c6 P a0a4efea0cec47928cd91fe2ec6ce02e: Log is configured to *not* fsync() on all Append() calls
I20260812 02:37:05.670209 17592 tablet_bootstrap.cc:492] T 8c8abcd1a2874017bbc453addabcf6c6 P 68b44fe628d849aea0058efe05f83434: No bootstrap required, opened a new log
I20260812 02:37:05.670542 17592 ts_tablet_manager.cc:1397] T 8c8abcd1a2874017bbc453addabcf6c6 P 68b44fe628d849aea0058efe05f83434: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260812 02:37:05.673012 17593 tablet_bootstrap.cc:492] T 8c8abcd1a2874017bbc453addabcf6c6 P a0a4efea0cec47928cd91fe2ec6ce02e: No bootstrap required, opened a new log
I20260812 02:37:05.673254 17593 ts_tablet_manager.cc:1397] T 8c8abcd1a2874017bbc453addabcf6c6 P a0a4efea0cec47928cd91fe2ec6ce02e: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260812 02:37:05.680022 17591 raft_consensus.cc:348] T 8c8abcd1a2874017bbc453addabcf6c6 P e97688088d8542f7a7d708d86b197530 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68b44fe628d849aea0058efe05f83434" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33969 } } peers { permanent_uuid: "a0a4efea0cec47928cd91fe2ec6ce02e" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34695 } } peers { permanent_uuid: "e97688088d8542f7a7d708d86b197530" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 42457 } }
I20260812 02:37:05.680450 17591 raft_consensus.cc:374] T 8c8abcd1a2874017bbc453addabcf6c6 P e97688088d8542f7a7d708d86b197530 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:37:05.680568 17591 raft_consensus.cc:729] T 8c8abcd1a2874017bbc453addabcf6c6 P e97688088d8542f7a7d708d86b197530 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e97688088d8542f7a7d708d86b197530, State: Initialized, Role: FOLLOWER
I20260812 02:37:05.680402 17593 raft_consensus.cc:348] T 8c8abcd1a2874017bbc453addabcf6c6 P a0a4efea0cec47928cd91fe2ec6ce02e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68b44fe628d849aea0058efe05f83434" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33969 } } peers { permanent_uuid: "a0a4efea0cec47928cd91fe2ec6ce02e" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34695 } } peers { permanent_uuid: "e97688088d8542f7a7d708d86b197530" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 42457 } }
I20260812 02:37:05.680812 17593 raft_consensus.cc:374] T 8c8abcd1a2874017bbc453addabcf6c6 P a0a4efea0cec47928cd91fe2ec6ce02e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:37:05.680927 17593 raft_consensus.cc:729] T 8c8abcd1a2874017bbc453addabcf6c6 P a0a4efea0cec47928cd91fe2ec6ce02e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0a4efea0cec47928cd91fe2ec6ce02e, State: Initialized, Role: FOLLOWER
I20260812 02:37:05.680907 17592 raft_consensus.cc:348] T 8c8abcd1a2874017bbc453addabcf6c6 P 68b44fe628d849aea0058efe05f83434 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68b44fe628d849aea0058efe05f83434" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33969 } } peers { permanent_uuid: "a0a4efea0cec47928cd91fe2ec6ce02e" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34695 } } peers { permanent_uuid: "e97688088d8542f7a7d708d86b197530" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 42457 } }
I20260812 02:37:05.681308 17592 raft_consensus.cc:374] T 8c8abcd1a2874017bbc453addabcf6c6 P 68b44fe628d849aea0058efe05f83434 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:37:05.681202 17591 consensus_queue.cc:260] T 8c8abcd1a2874017bbc453addabcf6c6 P e97688088d8542f7a7d708d86b197530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68b44fe628d849aea0058efe05f83434" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33969 } } peers { permanent_uuid: "a0a4efea0cec47928cd91fe2ec6ce02e" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34695 } } peers { permanent_uuid: "e97688088d8542f7a7d708d86b197530" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 42457 } }
I20260812 02:37:05.681409 17592 raft_consensus.cc:729] T 8c8abcd1a2874017bbc453addabcf6c6 P 68b44fe628d849aea0058efe05f83434 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68b44fe628d849aea0058efe05f83434, State: Initialized, Role: FOLLOWER
I20260812 02:37:05.681515 17593 consensus_queue.cc:260] T 8c8abcd1a2874017bbc453addabcf6c6 P a0a4efea0cec47928cd91fe2ec6ce02e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68b44fe628d849aea0058efe05f83434" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33969 } } peers { permanent_uuid: "a0a4efea0cec47928cd91fe2ec6ce02e" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34695 } } peers { permanent_uuid: "e97688088d8542f7a7d708d86b197530" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 42457 } }
I20260812 02:37:05.683065 17591 ts_tablet_manager.cc:1428] T 8c8abcd1a2874017bbc453addabcf6c6 P e97688088d8542f7a7d708d86b197530: Time spent starting tablet: real 0.014s user 0.015s sys 0.000s
I20260812 02:37:05.684063 17592 consensus_queue.cc:260] T 8c8abcd1a2874017bbc453addabcf6c6 P 68b44fe628d849aea0058efe05f83434 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68b44fe628d849aea0058efe05f83434" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33969 } } peers { permanent_uuid: "a0a4efea0cec47928cd91fe2ec6ce02e" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34695 } } peers { permanent_uuid: "e97688088d8542f7a7d708d86b197530" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 42457 } }
I20260812 02:37:05.685667 17592 ts_tablet_manager.cc:1428] T 8c8abcd1a2874017bbc453addabcf6c6 P 68b44fe628d849aea0058efe05f83434: Time spent starting tablet: real 0.015s user 0.012s sys 0.000s
I20260812 02:37:05.686884 17593 ts_tablet_manager.cc:1428] T 8c8abcd1a2874017bbc453addabcf6c6 P a0a4efea0cec47928cd91fe2ec6ce02e: Time spent starting tablet: real 0.014s user 0.008s sys 0.004s
I20260812 02:37:05.708336 17599 raft_consensus.cc:482] T 8c8abcd1a2874017bbc453addabcf6c6 P 68b44fe628d849aea0058efe05f83434 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 02:37:05.708606 17599 raft_consensus.cc:504] T 8c8abcd1a2874017bbc453addabcf6c6 P 68b44fe628d849aea0058efe05f83434 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68b44fe628d849aea0058efe05f83434" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33969 } } peers { permanent_uuid: "a0a4efea0cec47928cd91fe2ec6ce02e" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34695 } } peers { permanent_uuid: "e97688088d8542f7a7d708d86b197530" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 42457 } }
I20260812 02:37:05.710325 17599 leader_election.cc:288] T 8c8abcd1a2874017bbc453addabcf6c6 P 68b44fe628d849aea0058efe05f83434 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0a4efea0cec47928cd91fe2ec6ce02e (127.7.132.130:34695), e97688088d8542f7a7d708d86b197530 (127.7.132.129:42457)
I20260812 02:37:05.719193 17590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:05.711611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:41907 (local address 127.7.132.129:42457)
0812 02:37:05.711828 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:05.711835 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:05.712374 (+ 539us) server_negotiation.cc:407] Connection header received
0812 02:37:05.712557 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:05.712561 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:05.712666 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:05.712767 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:37:05.713266 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:05.714243 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:05.716583 (+ 2340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:05.717878 (+ 1295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:05.718048 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:05.718157 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0812 02:37:05.718806 (+ 649us) server_negotiation.cc:299] Negotiation successful
0812 02:37:05.718940 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 02:37:05.719214 17587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:05.710737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:33645 (local address 127.7.132.130:34695)
0812 02:37:05.710998 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:05.711008 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:05.711400 (+ 392us) server_negotiation.cc:407] Connection header received
0812 02:37:05.711691 (+ 291us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:05.711700 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:05.711840 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:05.711947 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:37:05.712676 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:05.713783 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:05.716573 (+ 2790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:05.717900 (+ 1327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:05.718051 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:05.718153 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0812 02:37:05.718806 (+ 653us) server_negotiation.cc:299] Negotiation successful
0812 02:37:05.718977 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 02:37:05.719337 17602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:05.710658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.130:34695 (local address 127.7.132.131:33645)
0812 02:37:05.711296 (+ 638us) negotiation.cc:107] Waiting for socket to connect
0812 02:37:05.711311 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 02:37:05.711494 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:37:05.711991 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:37:05.711996 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:37:05.712031 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:37:05.712519 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:05.712531 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:05.713921 (+ 1390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:05.713928 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:05.716386 (+ 2458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:05.716407 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:05.718148 (+ 1741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:05.718199 (+ 51us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:05.718485 (+ 286us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:05.718564 (+ 79us) client_negotiation.cc:769] Sending connection context
0812 02:37:05.718658 (+ 94us) client_negotiation.cc:240] Negotiation successful
0812 02:37:05.718840 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":528,"mutex_wait_us":150,"thread_start_us":370,"threads_started":1}
I20260812 02:37:05.719571 17603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:05.711452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:42457 (local address 127.7.132.131:41907)
0812 02:37:05.712262 (+ 810us) negotiation.cc:107] Waiting for socket to connect
0812 02:37:05.712284 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 02:37:05.712423 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:37:05.712773 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:37:05.712776 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:37:05.712794 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:37:05.713105 (+ 311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:05.713120 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:05.714370 (+ 1250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:05.714375 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:05.716437 (+ 2062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:05.716449 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:05.718179 (+ 1730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:05.718184 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:05.718485 (+ 301us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:05.718553 (+ 68us) client_negotiation.cc:769] Sending connection context
0812 02:37:05.718658 (+ 105us) client_negotiation.cc:240] Negotiation successful
0812 02:37:05.718840 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":650,"mutex_wait_us":74,"thread_start_us":382,"threads_started":1}
I20260812 02:37:05.720412 17349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c8abcd1a2874017bbc453addabcf6c6" candidate_uuid: "68b44fe628d849aea0058efe05f83434" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0a4efea0cec47928cd91fe2ec6ce02e" is_pre_election: true
I20260812 02:37:05.720484 17180 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c8abcd1a2874017bbc453addabcf6c6" candidate_uuid: "68b44fe628d849aea0058efe05f83434" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e97688088d8542f7a7d708d86b197530" is_pre_election: true
I20260812 02:37:05.720971 17180 raft_consensus.cc:2454] T 8c8abcd1a2874017bbc453addabcf6c6 P e97688088d8542f7a7d708d86b197530 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68b44fe628d849aea0058efe05f83434 in term 0.
I20260812 02:37:05.720956 17349 raft_consensus.cc:2454] T 8c8abcd1a2874017bbc453addabcf6c6 P a0a4efea0cec47928cd91fe2ec6ce02e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68b44fe628d849aea0058efe05f83434 in term 0.
I20260812 02:37:05.721680 17425 leader_election.cc:302] T 8c8abcd1a2874017bbc453addabcf6c6 P 68b44fe628d849aea0058efe05f83434 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68b44fe628d849aea0058efe05f83434, e97688088d8542f7a7d708d86b197530; no voters:
I20260812 02:37:05.722146 17599 raft_consensus.cc:2781] T 8c8abcd1a2874017bbc453addabcf6c6 P 68b44fe628d849aea0058efe05f83434 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 02:37:05.722326 17599 raft_consensus.cc:482] T 8c8abcd1a2874017bbc453addabcf6c6 P 68b44fe628d849aea0058efe05f83434 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 02:37:05.722445 17599 raft_consensus.cc:3037] T 8c8abcd1a2874017bbc453addabcf6c6 P 68b44fe628d849aea0058efe05f83434 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:37:05.724658 17599 raft_consensus.cc:504] T 8c8abcd1a2874017bbc453addabcf6c6 P 68b44fe628d849aea0058efe05f83434 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68b44fe628d849aea0058efe05f83434" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33969 } } peers { permanent_uuid: "a0a4efea0cec47928cd91fe2ec6ce02e" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34695 } } peers { permanent_uuid: "e97688088d8542f7a7d708d86b197530" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 42457 } }
I20260812 02:37:05.725296 17599 leader_election.cc:288] T 8c8abcd1a2874017bbc453addabcf6c6 P 68b44fe628d849aea0058efe05f83434 [CANDIDATE]: Term 1 election: Requested vote from peers a0a4efea0cec47928cd91fe2ec6ce02e (127.7.132.130:34695), e97688088d8542f7a7d708d86b197530 (127.7.132.129:42457)
I20260812 02:37:05.725705 17349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c8abcd1a2874017bbc453addabcf6c6" candidate_uuid: "68b44fe628d849aea0058efe05f83434" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0a4efea0cec47928cd91fe2ec6ce02e"
I20260812 02:37:05.725765 17180 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c8abcd1a2874017bbc453addabcf6c6" candidate_uuid: "68b44fe628d849aea0058efe05f83434" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e97688088d8542f7a7d708d86b197530"
I20260812 02:37:05.725957 17180 raft_consensus.cc:3037] T 8c8abcd1a2874017bbc453addabcf6c6 P e97688088d8542f7a7d708d86b197530 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:37:05.725993 17349 raft_consensus.cc:3037] T 8c8abcd1a2874017bbc453addabcf6c6 P a0a4efea0cec47928cd91fe2ec6ce02e [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:37:05.728185 17180 raft_consensus.cc:2454] T 8c8abcd1a2874017bbc453addabcf6c6 P e97688088d8542f7a7d708d86b197530 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68b44fe628d849aea0058efe05f83434 in term 1.
I20260812 02:37:05.728549 17425 leader_election.cc:302] T 8c8abcd1a2874017bbc453addabcf6c6 P 68b44fe628d849aea0058efe05f83434 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68b44fe628d849aea0058efe05f83434, e97688088d8542f7a7d708d86b197530; no voters:
I20260812 02:37:05.728598 17349 raft_consensus.cc:2454] T 8c8abcd1a2874017bbc453addabcf6c6 P a0a4efea0cec47928cd91fe2ec6ce02e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68b44fe628d849aea0058efe05f83434 in term 1.
I20260812 02:37:05.728852 17599 raft_consensus.cc:2781] T 8c8abcd1a2874017bbc453addabcf6c6 P 68b44fe628d849aea0058efe05f83434 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:37:05.729348 17599 raft_consensus.cc:686] T 8c8abcd1a2874017bbc453addabcf6c6 P 68b44fe628d849aea0058efe05f83434 [term 1 LEADER]: Becoming Leader. State: Replica: 68b44fe628d849aea0058efe05f83434, State: Running, Role: LEADER
I20260812 02:37:05.729912 17599 consensus_queue.cc:237] T 8c8abcd1a2874017bbc453addabcf6c6 P 68b44fe628d849aea0058efe05f83434 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68b44fe628d849aea0058efe05f83434" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33969 } } peers { permanent_uuid: "a0a4efea0cec47928cd91fe2ec6ce02e" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34695 } } peers { permanent_uuid: "e97688088d8542f7a7d708d86b197530" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 42457 } }
I20260812 02:37:05.734707 16997 catalog_manager.cc:5495] T 8c8abcd1a2874017bbc453addabcf6c6 P 68b44fe628d849aea0058efe05f83434 reported cstate change: term changed from 0 to 1, leader changed from <none> to 68b44fe628d849aea0058efe05f83434 (127.7.132.131). New cstate: current_term: 1 leader_uuid: "68b44fe628d849aea0058efe05f83434" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68b44fe628d849aea0058efe05f83434" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33969 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0a4efea0cec47928cd91fe2ec6ce02e" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34695 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e97688088d8542f7a7d708d86b197530" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 42457 } health_report { overall_health: UNKNOWN } } }
I20260812 02:37:05.757684 17588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:05.752738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59388 (local address 127.7.132.131:33969)
0812 02:37:05.752921 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:05.752926 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:05.752943 (+ 17us) server_negotiation.cc:407] Connection header received
0812 02:37:05.753024 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:05.753027 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:05.753110 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:05.753236 (+ 126us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 02:37:05.753552 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:05.754342 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:05.755594 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:05.756030 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:05.756123 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:05.756514 (+ 391us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 02:37:05.757060 (+ 546us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 02:37:05.757168 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0812 02:37:05.757270 (+ 102us) server_negotiation.cc:299] Negotiation successful
0812 02:37:05.757371 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260812 02:37:05.765801 17180 raft_consensus.cc:1261] T 8c8abcd1a2874017bbc453addabcf6c6 P e97688088d8542f7a7d708d86b197530 [term 1 FOLLOWER]: Refusing update from remote peer 68b44fe628d849aea0058efe05f83434: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:37:05.765789 17349 raft_consensus.cc:1261] T 8c8abcd1a2874017bbc453addabcf6c6 P a0a4efea0cec47928cd91fe2ec6ce02e [term 1 FOLLOWER]: Refusing update from remote peer 68b44fe628d849aea0058efe05f83434: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:37:05.766770 17599 consensus_queue.cc:1035] T 8c8abcd1a2874017bbc453addabcf6c6 P 68b44fe628d849aea0058efe05f83434 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0a4efea0cec47928cd91fe2ec6ce02e" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:37:05.767095 17604 consensus_queue.cc:1035] T 8c8abcd1a2874017bbc453addabcf6c6 P 68b44fe628d849aea0058efe05f83434 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e97688088d8542f7a7d708d86b197530" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 42457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 02:37:05.775888 17239 tablet.cc:2369] T 8c8abcd1a2874017bbc453addabcf6c6 P e97688088d8542f7a7d708d86b197530: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 02:37:05.798309 16997 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44856:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 02:37:05.798593 16997 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44856:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 02:37:05.802461 16997 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1c4324eb32464053b83080be530c21b3: Sending DeleteTablet for 3 replicas of tablet 8c8abcd1a2874017bbc453addabcf6c6
I20260812 02:37:05.803666 17484 tablet_service.cc:1514] Processing DeleteTablet for tablet 8c8abcd1a2874017bbc453addabcf6c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:37:05 UTC) from {username='oryx', principal='oryx/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:59376
I20260812 02:37:05.803781 17320 tablet_service.cc:1514] Processing DeleteTablet for tablet 8c8abcd1a2874017bbc453addabcf6c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:37:05 UTC) from {username='oryx', principal='oryx/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:44494
I20260812 02:37:05.803956 17156 tablet_service.cc:1514] Processing DeleteTablet for tablet 8c8abcd1a2874017bbc453addabcf6c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:37:05 UTC) from {username='oryx', principal='oryx/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:36814
I20260812 02:37:05.806298 16997 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44856:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 02:37:05.807212 16997 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 02:37:05.818789 17319 tablet_service.cc:1467] Processing CreateTablet for tablet 7f7a41b7cd7f456db6d257ee485f2473 (DEFAULT_TABLE table=test-table [id=721127c7f06a4fcb8b8f5dabcf10f2de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:37:05.819542 17319 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f7a41b7cd7f456db6d257ee485f2473. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:37:05.822950 17483 tablet_service.cc:1467] Processing CreateTablet for tablet 7f7a41b7cd7f456db6d257ee485f2473 (DEFAULT_TABLE table=test-table [id=721127c7f06a4fcb8b8f5dabcf10f2de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:37:05.823405 17155 tablet_service.cc:1467] Processing CreateTablet for tablet 7f7a41b7cd7f456db6d257ee485f2473 (DEFAULT_TABLE table=test-table [id=721127c7f06a4fcb8b8f5dabcf10f2de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:37:05.824113 17155 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f7a41b7cd7f456db6d257ee485f2473. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:37:05.824755 17621 tablet_replica.cc:330] T 8c8abcd1a2874017bbc453addabcf6c6 P e97688088d8542f7a7d708d86b197530: stopping tablet replica
I20260812 02:37:05.825416 17621 raft_consensus.cc:2229] T 8c8abcd1a2874017bbc453addabcf6c6 P e97688088d8542f7a7d708d86b197530 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 02:37:05.826067 17621 raft_consensus.cc:2258] T 8c8abcd1a2874017bbc453addabcf6c6 P e97688088d8542f7a7d708d86b197530 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:37:05.827589 17618 tablet_replica.cc:330] T 8c8abcd1a2874017bbc453addabcf6c6 P a0a4efea0cec47928cd91fe2ec6ce02e: stopping tablet replica
I20260812 02:37:05.828027 17618 raft_consensus.cc:2229] T 8c8abcd1a2874017bbc453addabcf6c6 P a0a4efea0cec47928cd91fe2ec6ce02e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 02:37:05.828220 17593 tablet_bootstrap.cc:492] T 7f7a41b7cd7f456db6d257ee485f2473 P a0a4efea0cec47928cd91fe2ec6ce02e: Bootstrap starting.
I20260812 02:37:05.828588 17618 raft_consensus.cc:2258] T 8c8abcd1a2874017bbc453addabcf6c6 P a0a4efea0cec47928cd91fe2ec6ce02e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:37:05.829099 17619 tablet_replica.cc:330] T 8c8abcd1a2874017bbc453addabcf6c6 P 68b44fe628d849aea0058efe05f83434: stopping tablet replica
I20260812 02:37:05.829526 17619 raft_consensus.cc:2229] T 8c8abcd1a2874017bbc453addabcf6c6 P 68b44fe628d849aea0058efe05f83434 [term 1 LEADER]: Raft consensus shutting down.
I20260812 02:37:05.829978 17593 tablet_bootstrap.cc:654] T 7f7a41b7cd7f456db6d257ee485f2473 P a0a4efea0cec47928cd91fe2ec6ce02e: Neither blocks nor log segments found. Creating new log.
I20260812 02:37:05.830253 17619 raft_consensus.cc:2258] T 8c8abcd1a2874017bbc453addabcf6c6 P 68b44fe628d849aea0058efe05f83434 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:37:05.832064 17618 ts_tablet_manager.cc:1905] T 8c8abcd1a2874017bbc453addabcf6c6 P a0a4efea0cec47928cd91fe2ec6ce02e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:37:05.832120 17591 tablet_bootstrap.cc:492] T 7f7a41b7cd7f456db6d257ee485f2473 P e97688088d8542f7a7d708d86b197530: Bootstrap starting.
I20260812 02:37:05.832144 17483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f7a41b7cd7f456db6d257ee485f2473. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:37:05.833777 17621 ts_tablet_manager.cc:1905] T 8c8abcd1a2874017bbc453addabcf6c6 P e97688088d8542f7a7d708d86b197530: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:37:05.833827 17591 tablet_bootstrap.cc:654] T 7f7a41b7cd7f456db6d257ee485f2473 P e97688088d8542f7a7d708d86b197530: Neither blocks nor log segments found. Creating new log.
I20260812 02:37:05.840938 17621 ts_tablet_manager.cc:1918] T 8c8abcd1a2874017bbc453addabcf6c6 P e97688088d8542f7a7d708d86b197530: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 02:37:05.841136 17621 log.cc:1196] T 8c8abcd1a2874017bbc453addabcf6c6 P e97688088d8542f7a7d708d86b197530: Deleting WAL directory at /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502156971023-7698-0/minicluster-data/ts-0/wal/wals/8c8abcd1a2874017bbc453addabcf6c6
I20260812 02:37:05.841662 17621 ts_tablet_manager.cc:1939] T 8c8abcd1a2874017bbc453addabcf6c6 P e97688088d8542f7a7d708d86b197530: Deleting consensus metadata
I20260812 02:37:05.843459 17592 tablet_bootstrap.cc:492] T 7f7a41b7cd7f456db6d257ee485f2473 P 68b44fe628d849aea0058efe05f83434: Bootstrap starting.
I20260812 02:37:05.843757 16975 catalog_manager.cc:4841] TS e97688088d8542f7a7d708d86b197530 (127.7.132.129:42457): tablet 8c8abcd1a2874017bbc453addabcf6c6 (table test-table [id=4d448d228ef047719c912472c4f46f85]) successfully deleted
I20260812 02:37:05.844372 17618 ts_tablet_manager.cc:1918] T 8c8abcd1a2874017bbc453addabcf6c6 P a0a4efea0cec47928cd91fe2ec6ce02e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 02:37:05.844537 17618 log.cc:1196] T 8c8abcd1a2874017bbc453addabcf6c6 P a0a4efea0cec47928cd91fe2ec6ce02e: Deleting WAL directory at /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502156971023-7698-0/minicluster-data/ts-1/wal/wals/8c8abcd1a2874017bbc453addabcf6c6
I20260812 02:37:05.845094 17618 ts_tablet_manager.cc:1939] T 8c8abcd1a2874017bbc453addabcf6c6 P a0a4efea0cec47928cd91fe2ec6ce02e: Deleting consensus metadata
I20260812 02:37:05.845541 17592 tablet_bootstrap.cc:654] T 7f7a41b7cd7f456db6d257ee485f2473 P 68b44fe628d849aea0058efe05f83434: Neither blocks nor log segments found. Creating new log.
I20260812 02:37:05.846001 17619 ts_tablet_manager.cc:1905] T 8c8abcd1a2874017bbc453addabcf6c6 P 68b44fe628d849aea0058efe05f83434: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:37:05.846874 16975 catalog_manager.cc:4841] TS a0a4efea0cec47928cd91fe2ec6ce02e (127.7.132.130:34695): tablet 8c8abcd1a2874017bbc453addabcf6c6 (table test-table [id=4d448d228ef047719c912472c4f46f85]) successfully deleted
I20260812 02:37:05.853051 17619 ts_tablet_manager.cc:1918] T 8c8abcd1a2874017bbc453addabcf6c6 P 68b44fe628d849aea0058efe05f83434: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 02:37:05.853229 17619 log.cc:1196] T 8c8abcd1a2874017bbc453addabcf6c6 P 68b44fe628d849aea0058efe05f83434: Deleting WAL directory at /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502156971023-7698-0/minicluster-data/ts-2/wal/wals/8c8abcd1a2874017bbc453addabcf6c6
I20260812 02:37:05.853744 17619 ts_tablet_manager.cc:1939] T 8c8abcd1a2874017bbc453addabcf6c6 P 68b44fe628d849aea0058efe05f83434: Deleting consensus metadata
I20260812 02:37:05.853909 17591 tablet_bootstrap.cc:492] T 7f7a41b7cd7f456db6d257ee485f2473 P e97688088d8542f7a7d708d86b197530: No bootstrap required, opened a new log
I20260812 02:37:05.854068 17591 ts_tablet_manager.cc:1397] T 7f7a41b7cd7f456db6d257ee485f2473 P e97688088d8542f7a7d708d86b197530: Time spent bootstrapping tablet: real 0.022s user 0.001s sys 0.010s
I20260812 02:37:05.854972 17591 raft_consensus.cc:348] T 7f7a41b7cd7f456db6d257ee485f2473 P e97688088d8542f7a7d708d86b197530 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e97688088d8542f7a7d708d86b197530" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 42457 } } peers { permanent_uuid: "a0a4efea0cec47928cd91fe2ec6ce02e" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34695 } } peers { permanent_uuid: "68b44fe628d849aea0058efe05f83434" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33969 } }
I20260812 02:37:05.855208 17591 raft_consensus.cc:374] T 7f7a41b7cd7f456db6d257ee485f2473 P e97688088d8542f7a7d708d86b197530 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:37:05.855309 17591 raft_consensus.cc:729] T 7f7a41b7cd7f456db6d257ee485f2473 P e97688088d8542f7a7d708d86b197530 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e97688088d8542f7a7d708d86b197530, State: Initialized, Role: FOLLOWER
I20260812 02:37:05.855331 16980 catalog_manager.cc:4841] TS 68b44fe628d849aea0058efe05f83434 (127.7.132.131:33969): tablet 8c8abcd1a2874017bbc453addabcf6c6 (table test-table [id=4d448d228ef047719c912472c4f46f85]) successfully deleted
I20260812 02:37:05.856105 17591 consensus_queue.cc:260] T 7f7a41b7cd7f456db6d257ee485f2473 P e97688088d8542f7a7d708d86b197530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e97688088d8542f7a7d708d86b197530" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 42457 } } peers { permanent_uuid: "a0a4efea0cec47928cd91fe2ec6ce02e" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34695 } } peers { permanent_uuid: "68b44fe628d849aea0058efe05f83434" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33969 } }
I20260812 02:37:05.856617 17592 tablet_bootstrap.cc:492] T 7f7a41b7cd7f456db6d257ee485f2473 P 68b44fe628d849aea0058efe05f83434: No bootstrap required, opened a new log
I20260812 02:37:05.856769 17592 ts_tablet_manager.cc:1397] T 7f7a41b7cd7f456db6d257ee485f2473 P 68b44fe628d849aea0058efe05f83434: Time spent bootstrapping tablet: real 0.013s user 0.006s sys 0.000s
I20260812 02:37:05.857590 17592 raft_consensus.cc:348] T 7f7a41b7cd7f456db6d257ee485f2473 P 68b44fe628d849aea0058efe05f83434 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e97688088d8542f7a7d708d86b197530" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 42457 } } peers { permanent_uuid: "a0a4efea0cec47928cd91fe2ec6ce02e" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34695 } } peers { permanent_uuid: "68b44fe628d849aea0058efe05f83434" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33969 } }
I20260812 02:37:05.857803 17592 raft_consensus.cc:374] T 7f7a41b7cd7f456db6d257ee485f2473 P 68b44fe628d849aea0058efe05f83434 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:37:05.857892 17592 raft_consensus.cc:729] T 7f7a41b7cd7f456db6d257ee485f2473 P 68b44fe628d849aea0058efe05f83434 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68b44fe628d849aea0058efe05f83434, State: Initialized, Role: FOLLOWER
I20260812 02:37:05.857975 17591 ts_tablet_manager.cc:1428] T 7f7a41b7cd7f456db6d257ee485f2473 P e97688088d8542f7a7d708d86b197530: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260812 02:37:05.858187 17592 consensus_queue.cc:260] T 7f7a41b7cd7f456db6d257ee485f2473 P 68b44fe628d849aea0058efe05f83434 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e97688088d8542f7a7d708d86b197530" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 42457 } } peers { permanent_uuid: "a0a4efea0cec47928cd91fe2ec6ce02e" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34695 } } peers { permanent_uuid: "68b44fe628d849aea0058efe05f83434" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33969 } }
I20260812 02:37:05.859423 17593 tablet_bootstrap.cc:492] T 7f7a41b7cd7f456db6d257ee485f2473 P a0a4efea0cec47928cd91fe2ec6ce02e: No bootstrap required, opened a new log
I20260812 02:37:05.859593 17593 ts_tablet_manager.cc:1397] T 7f7a41b7cd7f456db6d257ee485f2473 P a0a4efea0cec47928cd91fe2ec6ce02e: Time spent bootstrapping tablet: real 0.031s user 0.008s sys 0.004s
I20260812 02:37:05.860685 17592 ts_tablet_manager.cc:1428] T 7f7a41b7cd7f456db6d257ee485f2473 P 68b44fe628d849aea0058efe05f83434: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260812 02:37:05.860612 17593 raft_consensus.cc:348] T 7f7a41b7cd7f456db6d257ee485f2473 P a0a4efea0cec47928cd91fe2ec6ce02e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e97688088d8542f7a7d708d86b197530" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 42457 } } peers { permanent_uuid: "a0a4efea0cec47928cd91fe2ec6ce02e" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34695 } } peers { permanent_uuid: "68b44fe628d849aea0058efe05f83434" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33969 } }
I20260812 02:37:05.860859 17593 raft_consensus.cc:374] T 7f7a41b7cd7f456db6d257ee485f2473 P a0a4efea0cec47928cd91fe2ec6ce02e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:37:05.860966 17593 raft_consensus.cc:729] T 7f7a41b7cd7f456db6d257ee485f2473 P a0a4efea0cec47928cd91fe2ec6ce02e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0a4efea0cec47928cd91fe2ec6ce02e, State: Initialized, Role: FOLLOWER
I20260812 02:37:05.861320 17593 consensus_queue.cc:260] T 7f7a41b7cd7f456db6d257ee485f2473 P a0a4efea0cec47928cd91fe2ec6ce02e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e97688088d8542f7a7d708d86b197530" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 42457 } } peers { permanent_uuid: "a0a4efea0cec47928cd91fe2ec6ce02e" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34695 } } peers { permanent_uuid: "68b44fe628d849aea0058efe05f83434" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33969 } }
I20260812 02:37:05.862126 17593 ts_tablet_manager.cc:1428] T 7f7a41b7cd7f456db6d257ee485f2473 P a0a4efea0cec47928cd91fe2ec6ce02e: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
W20260812 02:37:05.913868 17565 tablet.cc:2369] T 7f7a41b7cd7f456db6d257ee485f2473 P 68b44fe628d849aea0058efe05f83434: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 02:37:06.015391 17597 raft_consensus.cc:482] T 7f7a41b7cd7f456db6d257ee485f2473 P e97688088d8542f7a7d708d86b197530 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 02:37:06.015671 17597 raft_consensus.cc:504] T 7f7a41b7cd7f456db6d257ee485f2473 P e97688088d8542f7a7d708d86b197530 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e97688088d8542f7a7d708d86b197530" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 42457 } } peers { permanent_uuid: "a0a4efea0cec47928cd91fe2ec6ce02e" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34695 } } peers { permanent_uuid: "68b44fe628d849aea0058efe05f83434" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33969 } }
I20260812 02:37:06.017858 17597 leader_election.cc:288] T 7f7a41b7cd7f456db6d257ee485f2473 P e97688088d8542f7a7d708d86b197530 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0a4efea0cec47928cd91fe2ec6ce02e (127.7.132.130:34695), 68b44fe628d849aea0058efe05f83434 (127.7.132.131:33969)
I20260812 02:37:06.026575 17587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:06.018518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:42513 (local address 127.7.132.130:34695)
0812 02:37:06.018710 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:06.018717 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:06.019816 (+ 1099us) server_negotiation.cc:407] Connection header received
0812 02:37:06.020059 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:06.020067 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:06.020150 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:06.020201 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:37:06.021032 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:06.022221 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:06.024607 (+ 2386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:06.025508 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:06.025725 (+ 217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:06.025803 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 02:37:06.026137 (+ 334us) server_negotiation.cc:299] Negotiation successful
0812 02:37:06.026274 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260812 02:37:06.026633 17627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:06.018122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.130:34695 (local address 127.7.132.129:42513)
0812 02:37:06.019685 (+ 1563us) negotiation.cc:107] Waiting for socket to connect
0812 02:37:06.019710 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0812 02:37:06.019839 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:37:06.020378 (+ 539us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:37:06.020381 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:37:06.020401 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:37:06.020843 (+ 442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:06.020857 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:06.022382 (+ 1525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:06.022388 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:06.024464 (+ 2076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:06.024474 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:06.025743 (+ 1269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:06.025749 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:06.025931 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:06.025957 (+ 26us) client_negotiation.cc:769] Sending connection context
0812 02:37:06.026066 (+ 109us) client_negotiation.cc:240] Negotiation successful
0812 02:37:06.026241 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1374,"thread_start_us":1249,"threads_started":1}
I20260812 02:37:06.027274 17349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f7a41b7cd7f456db6d257ee485f2473" candidate_uuid: "e97688088d8542f7a7d708d86b197530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0a4efea0cec47928cd91fe2ec6ce02e" is_pre_election: true
I20260812 02:37:06.027560 17349 raft_consensus.cc:2454] T 7f7a41b7cd7f456db6d257ee485f2473 P a0a4efea0cec47928cd91fe2ec6ce02e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e97688088d8542f7a7d708d86b197530 in term 0.
I20260812 02:37:06.027724 17625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:06.018120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.131:33969 (local address 127.7.132.129:53509)
0812 02:37:06.019186 (+ 1066us) negotiation.cc:107] Waiting for socket to connect
0812 02:37:06.019208 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 02:37:06.019427 (+ 219us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:37:06.019966 (+ 539us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:37:06.019972 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:37:06.020002 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:37:06.020374 (+ 372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:06.020387 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:06.021800 (+ 1413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:06.021808 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:06.024223 (+ 2415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:06.024240 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:06.026219 (+ 1979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:06.026226 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:06.026436 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:06.026471 (+ 35us) client_negotiation.cc:769] Sending connection context
0812 02:37:06.027241 (+ 770us) client_negotiation.cc:240] Negotiation successful
0812 02:37:06.027393 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":926,"thread_start_us":749,"threads_started":1}
I20260812 02:37:06.027720 17588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:06.018483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:53509 (local address 127.7.132.131:33969)
0812 02:37:06.018725 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:06.018738 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:06.019307 (+ 569us) server_negotiation.cc:407] Connection header received
0812 02:37:06.019703 (+ 396us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:06.019710 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:06.019836 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:06.019944 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:37:06.020558 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:06.021661 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:06.024403 (+ 2742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:06.025772 (+ 1369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:06.025938 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:06.026013 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 02:37:06.027101 (+ 1088us) server_negotiation.cc:299] Negotiation successful
0812 02:37:06.027257 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 02:37:06.028110 17098 leader_election.cc:302] T 7f7a41b7cd7f456db6d257ee485f2473 P e97688088d8542f7a7d708d86b197530 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a0a4efea0cec47928cd91fe2ec6ce02e, e97688088d8542f7a7d708d86b197530; no voters:
I20260812 02:37:06.028605 17597 raft_consensus.cc:2781] T 7f7a41b7cd7f456db6d257ee485f2473 P e97688088d8542f7a7d708d86b197530 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 02:37:06.028730 17597 raft_consensus.cc:482] T 7f7a41b7cd7f456db6d257ee485f2473 P e97688088d8542f7a7d708d86b197530 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 02:37:06.028815 17597 raft_consensus.cc:3037] T 7f7a41b7cd7f456db6d257ee485f2473 P e97688088d8542f7a7d708d86b197530 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:37:06.029277 17510 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f7a41b7cd7f456db6d257ee485f2473" candidate_uuid: "e97688088d8542f7a7d708d86b197530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68b44fe628d849aea0058efe05f83434" is_pre_election: true
I20260812 02:37:06.029621 17510 raft_consensus.cc:2454] T 7f7a41b7cd7f456db6d257ee485f2473 P 68b44fe628d849aea0058efe05f83434 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e97688088d8542f7a7d708d86b197530 in term 0.
I20260812 02:37:06.032101 17597 raft_consensus.cc:504] T 7f7a41b7cd7f456db6d257ee485f2473 P e97688088d8542f7a7d708d86b197530 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e97688088d8542f7a7d708d86b197530" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 42457 } } peers { permanent_uuid: "a0a4efea0cec47928cd91fe2ec6ce02e" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34695 } } peers { permanent_uuid: "68b44fe628d849aea0058efe05f83434" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33969 } }
I20260812 02:37:06.032789 17597 leader_election.cc:288] T 7f7a41b7cd7f456db6d257ee485f2473 P e97688088d8542f7a7d708d86b197530 [CANDIDATE]: Term 1 election: Requested vote from peers a0a4efea0cec47928cd91fe2ec6ce02e (127.7.132.130:34695), 68b44fe628d849aea0058efe05f83434 (127.7.132.131:33969)
I20260812 02:37:06.033283 17510 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f7a41b7cd7f456db6d257ee485f2473" candidate_uuid: "e97688088d8542f7a7d708d86b197530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68b44fe628d849aea0058efe05f83434"
I20260812 02:37:06.033318 17349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f7a41b7cd7f456db6d257ee485f2473" candidate_uuid: "e97688088d8542f7a7d708d86b197530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0a4efea0cec47928cd91fe2ec6ce02e"
I20260812 02:37:06.033499 17510 raft_consensus.cc:3037] T 7f7a41b7cd7f456db6d257ee485f2473 P 68b44fe628d849aea0058efe05f83434 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:37:06.033535 17349 raft_consensus.cc:3037] T 7f7a41b7cd7f456db6d257ee485f2473 P a0a4efea0cec47928cd91fe2ec6ce02e [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:37:06.035887 17510 raft_consensus.cc:2454] T 7f7a41b7cd7f456db6d257ee485f2473 P 68b44fe628d849aea0058efe05f83434 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e97688088d8542f7a7d708d86b197530 in term 1.
I20260812 02:37:06.036396 17103 leader_election.cc:302] T 7f7a41b7cd7f456db6d257ee485f2473 P e97688088d8542f7a7d708d86b197530 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68b44fe628d849aea0058efe05f83434, e97688088d8542f7a7d708d86b197530; no voters:
I20260812 02:37:06.036443 17349 raft_consensus.cc:2454] T 7f7a41b7cd7f456db6d257ee485f2473 P a0a4efea0cec47928cd91fe2ec6ce02e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e97688088d8542f7a7d708d86b197530 in term 1.
I20260812 02:37:06.036796 17597 raft_consensus.cc:2781] T 7f7a41b7cd7f456db6d257ee485f2473 P e97688088d8542f7a7d708d86b197530 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:37:06.037387 17597 raft_consensus.cc:686] T 7f7a41b7cd7f456db6d257ee485f2473 P e97688088d8542f7a7d708d86b197530 [term 1 LEADER]: Becoming Leader. State: Replica: e97688088d8542f7a7d708d86b197530, State: Running, Role: LEADER
I20260812 02:37:06.037811 17597 consensus_queue.cc:237] T 7f7a41b7cd7f456db6d257ee485f2473 P e97688088d8542f7a7d708d86b197530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e97688088d8542f7a7d708d86b197530" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 42457 } } peers { permanent_uuid: "a0a4efea0cec47928cd91fe2ec6ce02e" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34695 } } peers { permanent_uuid: "68b44fe628d849aea0058efe05f83434" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33969 } }
I20260812 02:37:06.041615 16998 catalog_manager.cc:5495] T 7f7a41b7cd7f456db6d257ee485f2473 P e97688088d8542f7a7d708d86b197530 reported cstate change: term changed from 0 to 1, leader changed from <none> to e97688088d8542f7a7d708d86b197530 (127.7.132.129). New cstate: current_term: 1 leader_uuid: "e97688088d8542f7a7d708d86b197530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e97688088d8542f7a7d708d86b197530" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 42457 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0a4efea0cec47928cd91fe2ec6ce02e" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34695 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "68b44fe628d849aea0058efe05f83434" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33969 } health_report { overall_health: UNKNOWN } } }
W20260812 02:37:06.085260 17403 tablet.cc:2369] T 7f7a41b7cd7f456db6d257ee485f2473 P a0a4efea0cec47928cd91fe2ec6ce02e: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 12 02:37:06 dist-test-slave-mn4r krb5kdc[16940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502222, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.132.190@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM
I20260812 02:37:06.100283 17433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:06.084827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44860 (local address 127.7.132.190:38167)
0812 02:37:06.085123 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:06.085133 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:06.085970 (+ 837us) server_negotiation.cc:407] Connection header received
0812 02:37:06.086256 (+ 286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:06.086260 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:06.086322 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:06.086427 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:37:06.087604 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:06.088340 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:06.089596 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:06.090016 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:06.090203 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:06.094169 (+ 3966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:37:06.094175 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:37:06.094179 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:37:06.094210 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:37:06.097191 (+ 2981us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:37:06.098179 (+ 988us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:37:06.098183 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:37:06.098185 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:37:06.098304 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:37:06.098773 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:37:06.098776 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:37:06.098778 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:37:06.099052 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:37:06.099187 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0812 02:37:06.099786 (+ 599us) server_negotiation.cc:299] Negotiation successful
0812 02:37:06.099990 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165}
I20260812 02:37:06.100692 17637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:06.084951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:38167 (local address 127.0.0.1:44860)
0812 02:37:06.085832 (+ 881us) negotiation.cc:107] Waiting for socket to connect
0812 02:37:06.085854 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 02:37:06.086040 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:37:06.086453 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:37:06.086458 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:37:06.087175 (+ 717us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:37:06.087456 (+ 281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:06.087466 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:06.088476 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:06.088484 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:06.089453 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:06.089465 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:06.090270 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:06.090275 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:06.090482 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:06.091266 (+ 784us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:37:06.091285 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:37:06.093908 (+ 2623us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:37:06.097347 (+ 3439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:37:06.097366 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:37:06.097375 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:37:06.098018 (+ 643us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:37:06.098424 (+ 406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:37:06.098431 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:37:06.098435 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:37:06.098656 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:37:06.099200 (+ 544us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:37:06.099210 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:37:06.099489 (+ 279us) client_negotiation.cc:769] Sending connection context
0812 02:37:06.099710 (+ 221us) client_negotiation.cc:240] Negotiation successful
0812 02:37:06.100196 (+ 486us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":732,"mutex_wait_us":129,"thread_start_us":407,"threads_started":1}
I20260812 02:37:06.106549 16998 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:44860:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260812 02:37:06.107734 16998 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 02:37:06.115778 17155 tablet_service.cc:1467] Processing CreateTablet for tablet 19eff56825f64dc1a0597604661e11f1 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=cd13da4272d54733bca8dad66845339d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 02:37:06.115849 17483 tablet_service.cc:1467] Processing CreateTablet for tablet 19eff56825f64dc1a0597604661e11f1 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=cd13da4272d54733bca8dad66845339d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 02:37:06.116442 17155 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19eff56825f64dc1a0597604661e11f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:37:06.116453 17483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19eff56825f64dc1a0597604661e11f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:37:06.122759 17319 tablet_service.cc:1467] Processing CreateTablet for tablet 19eff56825f64dc1a0597604661e11f1 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=cd13da4272d54733bca8dad66845339d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 02:37:06.123381 17319 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19eff56825f64dc1a0597604661e11f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:37:06.124408 17592 tablet_bootstrap.cc:492] T 19eff56825f64dc1a0597604661e11f1 P 68b44fe628d849aea0058efe05f83434: Bootstrap starting.
I20260812 02:37:06.126394 17592 tablet_bootstrap.cc:654] T 19eff56825f64dc1a0597604661e11f1 P 68b44fe628d849aea0058efe05f83434: Neither blocks nor log segments found. Creating new log.
I20260812 02:37:06.129166 17592 tablet_bootstrap.cc:492] T 19eff56825f64dc1a0597604661e11f1 P 68b44fe628d849aea0058efe05f83434: No bootstrap required, opened a new log
I20260812 02:37:06.129416 17592 ts_tablet_manager.cc:1397] T 19eff56825f64dc1a0597604661e11f1 P 68b44fe628d849aea0058efe05f83434: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.003s
I20260812 02:37:06.130148 17593 tablet_bootstrap.cc:492] T 19eff56825f64dc1a0597604661e11f1 P a0a4efea0cec47928cd91fe2ec6ce02e: Bootstrap starting.
I20260812 02:37:06.130429 17592 raft_consensus.cc:348] T 19eff56825f64dc1a0597604661e11f1 P 68b44fe628d849aea0058efe05f83434 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e97688088d8542f7a7d708d86b197530" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 42457 } } peers { permanent_uuid: "a0a4efea0cec47928cd91fe2ec6ce02e" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34695 } } peers { permanent_uuid: "68b44fe628d849aea0058efe05f83434" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33969 } }
I20260812 02:37:06.130710 17592 raft_consensus.cc:374] T 19eff56825f64dc1a0597604661e11f1 P 68b44fe628d849aea0058efe05f83434 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:37:06.130798 17592 raft_consensus.cc:729] T 19eff56825f64dc1a0597604661e11f1 P 68b44fe628d849aea0058efe05f83434 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68b44fe628d849aea0058efe05f83434, State: Initialized, Role: FOLLOWER
I20260812 02:37:06.131119 17592 consensus_queue.cc:260] T 19eff56825f64dc1a0597604661e11f1 P 68b44fe628d849aea0058efe05f83434 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e97688088d8542f7a7d708d86b197530" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 42457 } } peers { permanent_uuid: "a0a4efea0cec47928cd91fe2ec6ce02e" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34695 } } peers { permanent_uuid: "68b44fe628d849aea0058efe05f83434" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33969 } }
I20260812 02:37:06.131646 17599 tablet_replica.cc:439] T 19eff56825f64dc1a0597604661e11f1 P 68b44fe628d849aea0058efe05f83434: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e97688088d8542f7a7d708d86b197530" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 42457 } } peers { permanent_uuid: "a0a4efea0cec47928cd91fe2ec6ce02e" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34695 } } peers { permanent_uuid: "68b44fe628d849aea0058efe05f83434" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33969 } } }
I20260812 02:37:06.131909 17593 tablet_bootstrap.cc:654] T 19eff56825f64dc1a0597604661e11f1 P a0a4efea0cec47928cd91fe2ec6ce02e: Neither blocks nor log segments found. Creating new log.
I20260812 02:37:06.131932 17599 tablet_replica.cc:442] T 19eff56825f64dc1a0597604661e11f1 P 68b44fe628d849aea0058efe05f83434: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 02:37:06.133068 17591 tablet_bootstrap.cc:492] T 19eff56825f64dc1a0597604661e11f1 P e97688088d8542f7a7d708d86b197530: Bootstrap starting.
I20260812 02:37:06.134862 17592 ts_tablet_manager.cc:1428] T 19eff56825f64dc1a0597604661e11f1 P 68b44fe628d849aea0058efe05f83434: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20260812 02:37:06.134910 17591 tablet_bootstrap.cc:654] T 19eff56825f64dc1a0597604661e11f1 P e97688088d8542f7a7d708d86b197530: Neither blocks nor log segments found. Creating new log.
I20260812 02:37:06.136263 17593 tablet_bootstrap.cc:492] T 19eff56825f64dc1a0597604661e11f1 P a0a4efea0cec47928cd91fe2ec6ce02e: No bootstrap required, opened a new log
I20260812 02:37:06.136682 17593 ts_tablet_manager.cc:1397] T 19eff56825f64dc1a0597604661e11f1 P a0a4efea0cec47928cd91fe2ec6ce02e: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.002s
I20260812 02:37:06.137487 17593 raft_consensus.cc:348] T 19eff56825f64dc1a0597604661e11f1 P a0a4efea0cec47928cd91fe2ec6ce02e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e97688088d8542f7a7d708d86b197530" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 42457 } } peers { permanent_uuid: "a0a4efea0cec47928cd91fe2ec6ce02e" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34695 } } peers { permanent_uuid: "68b44fe628d849aea0058efe05f83434" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33969 } }
I20260812 02:37:06.137763 17593 raft_consensus.cc:374] T 19eff56825f64dc1a0597604661e11f1 P a0a4efea0cec47928cd91fe2ec6ce02e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:37:06.137844 17593 raft_consensus.cc:729] T 19eff56825f64dc1a0597604661e11f1 P a0a4efea0cec47928cd91fe2ec6ce02e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0a4efea0cec47928cd91fe2ec6ce02e, State: Initialized, Role: FOLLOWER
I20260812 02:37:06.138280 17593 consensus_queue.cc:260] T 19eff56825f64dc1a0597604661e11f1 P a0a4efea0cec47928cd91fe2ec6ce02e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e97688088d8542f7a7d708d86b197530" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 42457 } } peers { permanent_uuid: "a0a4efea0cec47928cd91fe2ec6ce02e" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34695 } } peers { permanent_uuid: "68b44fe628d849aea0058efe05f83434" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33969 } }
I20260812 02:37:06.138780 17598 tablet_replica.cc:439] T 19eff56825f64dc1a0597604661e11f1 P a0a4efea0cec47928cd91fe2ec6ce02e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e97688088d8542f7a7d708d86b197530" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 42457 } } peers { permanent_uuid: "a0a4efea0cec47928cd91fe2ec6ce02e" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34695 } } peers { permanent_uuid: "68b44fe628d849aea0058efe05f83434" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33969 } } }
I20260812 02:37:06.139184 17598 tablet_replica.cc:442] T 19eff56825f64dc1a0597604661e11f1 P a0a4efea0cec47928cd91fe2ec6ce02e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 02:37:06.141083 17593 ts_tablet_manager.cc:1428] T 19eff56825f64dc1a0597604661e11f1 P a0a4efea0cec47928cd91fe2ec6ce02e: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260812 02:37:06.147749 17591 tablet_bootstrap.cc:492] T 19eff56825f64dc1a0597604661e11f1 P e97688088d8542f7a7d708d86b197530: No bootstrap required, opened a new log
I20260812 02:37:06.147962 17591 ts_tablet_manager.cc:1397] T 19eff56825f64dc1a0597604661e11f1 P e97688088d8542f7a7d708d86b197530: Time spent bootstrapping tablet: real 0.015s user 0.003s sys 0.009s
I20260812 02:37:06.148811 17591 raft_consensus.cc:348] T 19eff56825f64dc1a0597604661e11f1 P e97688088d8542f7a7d708d86b197530 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e97688088d8542f7a7d708d86b197530" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 42457 } } peers { permanent_uuid: "a0a4efea0cec47928cd91fe2ec6ce02e" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34695 } } peers { permanent_uuid: "68b44fe628d849aea0058efe05f83434" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33969 } }
I20260812 02:37:06.149097 17591 raft_consensus.cc:374] T 19eff56825f64dc1a0597604661e11f1 P e97688088d8542f7a7d708d86b197530 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:37:06.149192 17591 raft_consensus.cc:729] T 19eff56825f64dc1a0597604661e11f1 P e97688088d8542f7a7d708d86b197530 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e97688088d8542f7a7d708d86b197530, State: Initialized, Role: FOLLOWER
I20260812 02:37:06.149466 17591 consensus_queue.cc:260] T 19eff56825f64dc1a0597604661e11f1 P e97688088d8542f7a7d708d86b197530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e97688088d8542f7a7d708d86b197530" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 42457 } } peers { permanent_uuid: "a0a4efea0cec47928cd91fe2ec6ce02e" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34695 } } peers { permanent_uuid: "68b44fe628d849aea0058efe05f83434" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33969 } }
I20260812 02:37:06.149938 17597 tablet_replica.cc:439] T 19eff56825f64dc1a0597604661e11f1 P e97688088d8542f7a7d708d86b197530: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e97688088d8542f7a7d708d86b197530" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 42457 } } peers { permanent_uuid: "a0a4efea0cec47928cd91fe2ec6ce02e" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34695 } } peers { permanent_uuid: "68b44fe628d849aea0058efe05f83434" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33969 } } }
I20260812 02:37:06.150277 17597 tablet_replica.cc:442] T 19eff56825f64dc1a0597604661e11f1 P e97688088d8542f7a7d708d86b197530: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 02:37:06.150744 17591 ts_tablet_manager.cc:1428] T 19eff56825f64dc1a0597604661e11f1 P e97688088d8542f7a7d708d86b197530: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260812 02:37:06.180665 17598 raft_consensus.cc:482] T 19eff56825f64dc1a0597604661e11f1 P a0a4efea0cec47928cd91fe2ec6ce02e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 02:37:06.180951 17598 raft_consensus.cc:504] T 19eff56825f64dc1a0597604661e11f1 P a0a4efea0cec47928cd91fe2ec6ce02e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e97688088d8542f7a7d708d86b197530" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 42457 } } peers { permanent_uuid: "a0a4efea0cec47928cd91fe2ec6ce02e" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34695 } } peers { permanent_uuid: "68b44fe628d849aea0058efe05f83434" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33969 } }
I20260812 02:37:06.182606 17598 leader_election.cc:288] T 19eff56825f64dc1a0597604661e11f1 P a0a4efea0cec47928cd91fe2ec6ce02e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e97688088d8542f7a7d708d86b197530 (127.7.132.129:42457), 68b44fe628d849aea0058efe05f83434 (127.7.132.131:33969)
I20260812 02:37:06.191207 17588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:06.183320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:56777 (local address 127.7.132.131:33969)
0812 02:37:06.183551 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:06.183558 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:06.184700 (+ 1142us) server_negotiation.cc:407] Connection header received
0812 02:37:06.184970 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:06.184975 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:06.185056 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:06.185127 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:37:06.185944 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:06.186793 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:06.188741 (+ 1948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:06.190014 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:06.190246 (+ 232us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:06.190324 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 02:37:06.190721 (+ 397us) server_negotiation.cc:299] Negotiation successful
0812 02:37:06.190876 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260812 02:37:06.191311 17642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:06.182900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.131:33969 (local address 127.7.132.130:56777)
0812 02:37:06.184601 (+ 1701us) negotiation.cc:107] Waiting for socket to connect
0812 02:37:06.184616 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 02:37:06.184779 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:37:06.185323 (+ 544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:37:06.185327 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:37:06.185354 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:37:06.185751 (+ 397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:06.185767 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:06.186951 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:06.186955 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:06.188570 (+ 1615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:06.188583 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:06.190287 (+ 1704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:06.190293 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:06.190486 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:06.190526 (+ 40us) client_negotiation.cc:769] Sending connection context
0812 02:37:06.190685 (+ 159us) client_negotiation.cc:240] Negotiation successful
0812 02:37:06.190848 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1570,"mutex_wait_us":7,"thread_start_us":1373,"threads_started":1}
I20260812 02:37:06.191963 17510 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19eff56825f64dc1a0597604661e11f1" candidate_uuid: "a0a4efea0cec47928cd91fe2ec6ce02e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68b44fe628d849aea0058efe05f83434" is_pre_election: true
I20260812 02:37:06.192233 17590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:06.183086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:59827 (local address 127.7.132.129:42457)
0812 02:37:06.183326 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:06.183335 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:06.183957 (+ 622us) server_negotiation.cc:407] Connection header received
0812 02:37:06.184184 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:06.184191 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:06.184292 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:06.184393 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:37:06.185267 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:06.186522 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:06.189249 (+ 2727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:06.190539 (+ 1290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:06.190750 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:06.190822 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 02:37:06.191797 (+ 975us) server_negotiation.cc:299] Negotiation successful
0812 02:37:06.191933 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260812 02:37:06.192265 17510 raft_consensus.cc:2454] T 19eff56825f64dc1a0597604661e11f1 P 68b44fe628d849aea0058efe05f83434 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0a4efea0cec47928cd91fe2ec6ce02e in term 0.
I20260812 02:37:06.192570 17641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:06.182823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:42457 (local address 127.7.132.130:59827)
0812 02:37:06.183843 (+ 1020us) negotiation.cc:107] Waiting for socket to connect
0812 02:37:06.183860 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 02:37:06.184027 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:37:06.184555 (+ 528us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:37:06.184560 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:37:06.184596 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:37:06.185033 (+ 437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:06.185049 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:06.186716 (+ 1667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:06.186722 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:06.189069 (+ 2347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:06.189086 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:06.191002 (+ 1916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:06.191009 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:06.191183 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:06.191224 (+ 41us) client_negotiation.cc:769] Sending connection context
0812 02:37:06.191770 (+ 546us) client_negotiation.cc:240] Negotiation successful
0812 02:37:06.191906 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":894,"mutex_wait_us":30,"thread_start_us":721,"threads_started":1}
I20260812 02:37:06.192838 17267 leader_election.cc:302] T 19eff56825f64dc1a0597604661e11f1 P a0a4efea0cec47928cd91fe2ec6ce02e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68b44fe628d849aea0058efe05f83434, a0a4efea0cec47928cd91fe2ec6ce02e; no voters:
I20260812 02:37:06.193285 17598 raft_consensus.cc:2781] T 19eff56825f64dc1a0597604661e11f1 P a0a4efea0cec47928cd91fe2ec6ce02e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 02:37:06.193334 17180 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19eff56825f64dc1a0597604661e11f1" candidate_uuid: "a0a4efea0cec47928cd91fe2ec6ce02e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e97688088d8542f7a7d708d86b197530" is_pre_election: true
I20260812 02:37:06.193424 17598 raft_consensus.cc:482] T 19eff56825f64dc1a0597604661e11f1 P a0a4efea0cec47928cd91fe2ec6ce02e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 02:37:06.193521 17598 raft_consensus.cc:3037] T 19eff56825f64dc1a0597604661e11f1 P a0a4efea0cec47928cd91fe2ec6ce02e [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:37:06.193596 17180 raft_consensus.cc:2454] T 19eff56825f64dc1a0597604661e11f1 P e97688088d8542f7a7d708d86b197530 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0a4efea0cec47928cd91fe2ec6ce02e in term 0.
I20260812 02:37:06.196054 17598 raft_consensus.cc:504] T 19eff56825f64dc1a0597604661e11f1 P a0a4efea0cec47928cd91fe2ec6ce02e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e97688088d8542f7a7d708d86b197530" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 42457 } } peers { permanent_uuid: "a0a4efea0cec47928cd91fe2ec6ce02e" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34695 } } peers { permanent_uuid: "68b44fe628d849aea0058efe05f83434" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33969 } }
I20260812 02:37:06.196663 17598 leader_election.cc:288] T 19eff56825f64dc1a0597604661e11f1 P a0a4efea0cec47928cd91fe2ec6ce02e [CANDIDATE]: Term 1 election: Requested vote from peers e97688088d8542f7a7d708d86b197530 (127.7.132.129:42457), 68b44fe628d849aea0058efe05f83434 (127.7.132.131:33969)
I20260812 02:37:06.197171 17180 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19eff56825f64dc1a0597604661e11f1" candidate_uuid: "a0a4efea0cec47928cd91fe2ec6ce02e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e97688088d8542f7a7d708d86b197530"
I20260812 02:37:06.197220 17510 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19eff56825f64dc1a0597604661e11f1" candidate_uuid: "a0a4efea0cec47928cd91fe2ec6ce02e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68b44fe628d849aea0058efe05f83434"
I20260812 02:37:06.197395 17180 raft_consensus.cc:3037] T 19eff56825f64dc1a0597604661e11f1 P e97688088d8542f7a7d708d86b197530 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:37:06.197448 17510 raft_consensus.cc:3037] T 19eff56825f64dc1a0597604661e11f1 P 68b44fe628d849aea0058efe05f83434 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:37:06.200445 17180 raft_consensus.cc:2454] T 19eff56825f64dc1a0597604661e11f1 P e97688088d8542f7a7d708d86b197530 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0a4efea0cec47928cd91fe2ec6ce02e in term 1.
I20260812 02:37:06.200888 17264 leader_election.cc:302] T 19eff56825f64dc1a0597604661e11f1 P a0a4efea0cec47928cd91fe2ec6ce02e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a0a4efea0cec47928cd91fe2ec6ce02e, e97688088d8542f7a7d708d86b197530; no voters:
I20260812 02:37:06.201210 17598 raft_consensus.cc:2781] T 19eff56825f64dc1a0597604661e11f1 P a0a4efea0cec47928cd91fe2ec6ce02e [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:37:06.200824 17510 raft_consensus.cc:2454] T 19eff56825f64dc1a0597604661e11f1 P 68b44fe628d849aea0058efe05f83434 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0a4efea0cec47928cd91fe2ec6ce02e in term 1.
I20260812 02:37:06.201822 17598 raft_consensus.cc:686] T 19eff56825f64dc1a0597604661e11f1 P a0a4efea0cec47928cd91fe2ec6ce02e [term 1 LEADER]: Becoming Leader. State: Replica: a0a4efea0cec47928cd91fe2ec6ce02e, State: Running, Role: LEADER
I20260812 02:37:06.202342 17598 consensus_queue.cc:237] T 19eff56825f64dc1a0597604661e11f1 P a0a4efea0cec47928cd91fe2ec6ce02e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e97688088d8542f7a7d708d86b197530" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 42457 } } peers { permanent_uuid: "a0a4efea0cec47928cd91fe2ec6ce02e" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34695 } } peers { permanent_uuid: "68b44fe628d849aea0058efe05f83434" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33969 } }
I20260812 02:37:06.205044 17644 tablet_replica.cc:439] T 19eff56825f64dc1a0597604661e11f1 P a0a4efea0cec47928cd91fe2ec6ce02e: TxnStatusTablet state changed. Reason: New leader a0a4efea0cec47928cd91fe2ec6ce02e. Latest consensus state: current_term: 1 leader_uuid: "a0a4efea0cec47928cd91fe2ec6ce02e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e97688088d8542f7a7d708d86b197530" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 42457 } } peers { permanent_uuid: "a0a4efea0cec47928cd91fe2ec6ce02e" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34695 } } peers { permanent_uuid: "68b44fe628d849aea0058efe05f83434" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33969 } } }
I20260812 02:37:06.205463 17644 tablet_replica.cc:442] T 19eff56825f64dc1a0597604661e11f1 P a0a4efea0cec47928cd91fe2ec6ce02e: This TxnStatusTablet replica's current role is: LEADER
I20260812 02:37:06.206235 17648 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260812 02:37:06.206483 17648 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260812 02:37:06.206529 16998 catalog_manager.cc:5495] T 19eff56825f64dc1a0597604661e11f1 P a0a4efea0cec47928cd91fe2ec6ce02e reported cstate change: term changed from 0 to 1, leader changed from <none> to a0a4efea0cec47928cd91fe2ec6ce02e (127.7.132.130). New cstate: current_term: 1 leader_uuid: "a0a4efea0cec47928cd91fe2ec6ce02e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e97688088d8542f7a7d708d86b197530" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 42457 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0a4efea0cec47928cd91fe2ec6ce02e" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34695 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "68b44fe628d849aea0058efe05f83434" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33969 } health_report { overall_health: UNKNOWN } } }
I20260812 02:37:06.236562 17637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:06.228688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.130:34695 (local address 127.0.0.1:44510)
0812 02:37:06.229074 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0812 02:37:06.229101 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0812 02:37:06.229266 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:37:06.229835 (+ 569us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:37:06.229841 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:37:06.229855 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 02:37:06.230102 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:06.230116 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:06.231661 (+ 1545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:06.231668 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:06.233361 (+ 1693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:06.233378 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:06.234446 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:06.234453 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:06.234685 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:06.234768 (+ 83us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 02:37:06.235910 (+ 1142us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 02:37:06.235945 (+ 35us) client_negotiation.cc:769] Sending connection context
0812 02:37:06.236047 (+ 102us) client_negotiation.cc:240] Negotiation successful
0812 02:37:06.236224 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192}
I20260812 02:37:06.236577 17587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:06.228778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44510 (local address 127.7.132.130:34695)
0812 02:37:06.228990 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:06.228997 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:06.229185 (+ 188us) server_negotiation.cc:407] Connection header received
0812 02:37:06.229499 (+ 314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:06.229505 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:06.229627 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:06.229791 (+ 164us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 02:37:06.230253 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:06.231489 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:06.233553 (+ 2064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:06.234272 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:06.234412 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:06.234998 (+ 586us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 02:37:06.235778 (+ 780us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 02:37:06.235929 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0812 02:37:06.236084 (+ 155us) server_negotiation.cc:299] Negotiation successful
0812 02:37:06.236181 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 02:37:06.241611 17180 raft_consensus.cc:1261] T 19eff56825f64dc1a0597604661e11f1 P e97688088d8542f7a7d708d86b197530 [term 1 FOLLOWER]: Refusing update from remote peer a0a4efea0cec47928cd91fe2ec6ce02e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:37:06.241737 17510 raft_consensus.cc:1261] T 19eff56825f64dc1a0597604661e11f1 P 68b44fe628d849aea0058efe05f83434 [term 1 FOLLOWER]: Refusing update from remote peer a0a4efea0cec47928cd91fe2ec6ce02e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:37:06.242502 17598 consensus_queue.cc:1035] T 19eff56825f64dc1a0597604661e11f1 P a0a4efea0cec47928cd91fe2ec6ce02e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e97688088d8542f7a7d708d86b197530" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 42457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:37:06.242908 17644 consensus_queue.cc:1035] T 19eff56825f64dc1a0597604661e11f1 P a0a4efea0cec47928cd91fe2ec6ce02e [LEADER]: Connected to new peer: Peer: permanent_uuid: "68b44fe628d849aea0058efe05f83434" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:37:06.255285 17597 tablet_replica.cc:439] T 19eff56825f64dc1a0597604661e11f1 P e97688088d8542f7a7d708d86b197530: TxnStatusTablet state changed. Reason: New leader a0a4efea0cec47928cd91fe2ec6ce02e. Latest consensus state: current_term: 1 leader_uuid: "a0a4efea0cec47928cd91fe2ec6ce02e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e97688088d8542f7a7d708d86b197530" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 42457 } } peers { permanent_uuid: "a0a4efea0cec47928cd91fe2ec6ce02e" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34695 } } peers { permanent_uuid: "68b44fe628d849aea0058efe05f83434" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33969 } } }
I20260812 02:37:06.255635 17597 tablet_replica.cc:442] T 19eff56825f64dc1a0597604661e11f1 P e97688088d8542f7a7d708d86b197530: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 02:37:06.256399 17599 tablet_replica.cc:439] T 19eff56825f64dc1a0597604661e11f1 P 68b44fe628d849aea0058efe05f83434: TxnStatusTablet state changed. Reason: New leader a0a4efea0cec47928cd91fe2ec6ce02e. Latest consensus state: current_term: 1 leader_uuid: "a0a4efea0cec47928cd91fe2ec6ce02e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e97688088d8542f7a7d708d86b197530" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 42457 } } peers { permanent_uuid: "a0a4efea0cec47928cd91fe2ec6ce02e" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34695 } } peers { permanent_uuid: "68b44fe628d849aea0058efe05f83434" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33969 } } }
I20260812 02:37:06.256716 17599 tablet_replica.cc:442] T 19eff56825f64dc1a0597604661e11f1 P 68b44fe628d849aea0058efe05f83434: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 02:37:06.259032 17598 tablet_replica.cc:439] T 19eff56825f64dc1a0597604661e11f1 P a0a4efea0cec47928cd91fe2ec6ce02e: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a0a4efea0cec47928cd91fe2ec6ce02e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e97688088d8542f7a7d708d86b197530" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 42457 } } peers { permanent_uuid: "a0a4efea0cec47928cd91fe2ec6ce02e" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34695 } } peers { permanent_uuid: "68b44fe628d849aea0058efe05f83434" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33969 } } }
I20260812 02:37:06.259342 17598 tablet_replica.cc:442] T 19eff56825f64dc1a0597604661e11f1 P a0a4efea0cec47928cd91fe2ec6ce02e: This TxnStatusTablet replica's current role is: LEADER
I20260812 02:37:06.259847 17608 mvcc.cc:205] Tried to move back new op lower bound from 7317513118677057536 to 7317513118529662976. Current Snapshot: MvccSnapshot[applied={T|T < 7317513118677057536}]
I20260812 02:37:06.266199 17599 tablet_replica.cc:439] T 19eff56825f64dc1a0597604661e11f1 P 68b44fe628d849aea0058efe05f83434: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a0a4efea0cec47928cd91fe2ec6ce02e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e97688088d8542f7a7d708d86b197530" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 42457 } } peers { permanent_uuid: "a0a4efea0cec47928cd91fe2ec6ce02e" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34695 } } peers { permanent_uuid: "68b44fe628d849aea0058efe05f83434" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33969 } } }
I20260812 02:37:06.266494 17599 tablet_replica.cc:442] T 19eff56825f64dc1a0597604661e11f1 P 68b44fe628d849aea0058efe05f83434: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 02:37:06.268841 17644 tablet_replica.cc:439] T 19eff56825f64dc1a0597604661e11f1 P a0a4efea0cec47928cd91fe2ec6ce02e: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a0a4efea0cec47928cd91fe2ec6ce02e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e97688088d8542f7a7d708d86b197530" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 42457 } } peers { permanent_uuid: "a0a4efea0cec47928cd91fe2ec6ce02e" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34695 } } peers { permanent_uuid: "68b44fe628d849aea0058efe05f83434" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33969 } } }
I20260812 02:37:06.269197 17644 tablet_replica.cc:442] T 19eff56825f64dc1a0597604661e11f1 P a0a4efea0cec47928cd91fe2ec6ce02e: This TxnStatusTablet replica's current role is: LEADER
I20260812 02:37:06.271149 17610 mvcc.cc:205] Tried to move back new op lower bound from 7317513118677057536 to 7317513118529662976. Current Snapshot: MvccSnapshot[applied={T|T < 7317513118677057536}]
I20260812 02:37:06.272724 17597 tablet_replica.cc:439] T 19eff56825f64dc1a0597604661e11f1 P e97688088d8542f7a7d708d86b197530: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a0a4efea0cec47928cd91fe2ec6ce02e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e97688088d8542f7a7d708d86b197530" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 42457 } } peers { permanent_uuid: "a0a4efea0cec47928cd91fe2ec6ce02e" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34695 } } peers { permanent_uuid: "68b44fe628d849aea0058efe05f83434" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33969 } } }
I20260812 02:37:06.273002 17597 tablet_replica.cc:442] T 19eff56825f64dc1a0597604661e11f1 P e97688088d8542f7a7d708d86b197530: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 02:37:06.324030 17590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:06.303786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59304 (local address 127.7.132.129:42457)
0812 02:37:06.303970 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:06.303976 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:06.316110 (+ 12134us) server_negotiation.cc:407] Connection header received
0812 02:37:06.316650 (+ 540us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:06.316657 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:06.316770 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:06.316924 (+ 154us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 02:37:06.317702 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:06.318991 (+ 1289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:06.321211 (+ 2220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:06.321927 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:06.322073 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:06.322576 (+ 503us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 02:37:06.323346 (+ 770us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 02:37:06.323496 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0812 02:37:06.323596 (+ 100us) server_negotiation.cc:299] Negotiation successful
0812 02:37:06.323692 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 02:37:06.341439 17587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:06.335626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:43441 (local address 127.7.132.130:34695)
0812 02:37:06.335830 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:06.335837 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:06.336480 (+ 643us) server_negotiation.cc:407] Connection header received
0812 02:37:06.336708 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:06.336713 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:06.336785 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:06.336890 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 02:37:06.337388 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:06.338197 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:06.339554 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:06.340046 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:06.340196 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:06.340628 (+ 432us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 02:37:06.340732 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 02:37:06.340833 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0812 02:37:06.340971 (+ 138us) server_negotiation.cc:299] Negotiation successful
0812 02:37:06.341053 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 02:37:06.341476 17661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:06.335592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.130:34695 (local address 127.7.132.129:43441)
0812 02:37:06.336372 (+ 780us) negotiation.cc:107] Waiting for socket to connect
0812 02:37:06.336388 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 02:37:06.336558 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:37:06.336905 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:37:06.336909 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:37:06.336921 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 02:37:06.337220 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:06.337229 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:06.338332 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:06.338338 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:06.339417 (+ 1079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:06.339429 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:06.340235 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:06.340243 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:06.340379 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:06.340432 (+ 53us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 02:37:06.340832 (+ 400us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 02:37:06.340844 (+ 12us) client_negotiation.cc:769] Sending connection context
0812 02:37:06.340945 (+ 101us) client_negotiation.cc:240] Negotiation successful
0812 02:37:06.341084 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":590,"thread_start_us":400,"threads_started":1}
I20260812 02:37:06.352025 17349 raft_consensus.cc:1261] T 7f7a41b7cd7f456db6d257ee485f2473 P a0a4efea0cec47928cd91fe2ec6ce02e [term 1 FOLLOWER]: Refusing update from remote peer e97688088d8542f7a7d708d86b197530: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:37:06.352084 17510 raft_consensus.cc:1261] T 7f7a41b7cd7f456db6d257ee485f2473 P 68b44fe628d849aea0058efe05f83434 [term 1 FOLLOWER]: Refusing update from remote peer e97688088d8542f7a7d708d86b197530: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:37:06.352728 17628 consensus_queue.cc:1035] T 7f7a41b7cd7f456db6d257ee485f2473 P e97688088d8542f7a7d708d86b197530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0a4efea0cec47928cd91fe2ec6ce02e" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:37:06.353060 17597 consensus_queue.cc:1035] T 7f7a41b7cd7f456db6d257ee485f2473 P e97688088d8542f7a7d708d86b197530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68b44fe628d849aea0058efe05f83434" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:37:06.390513 17590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:06.383622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:60811 (local address 127.7.132.129:42457)
0812 02:37:06.383791 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:06.383796 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:06.384618 (+ 822us) server_negotiation.cc:407] Connection header received
0812 02:37:06.384832 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:06.384837 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:06.384917 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:06.385008 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 02:37:06.385670 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:06.386623 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:06.388074 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:06.388587 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:06.388721 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:06.389221 (+ 500us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 02:37:06.389365 (+ 144us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 02:37:06.389474 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0812 02:37:06.390029 (+ 555us) server_negotiation.cc:299] Negotiation successful
0812 02:37:06.390143 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 02:37:06.390581 17667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:06.383719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:42457 (local address 127.7.132.130:60811)
0812 02:37:06.384515 (+ 796us) negotiation.cc:107] Waiting for socket to connect
0812 02:37:06.384537 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 02:37:06.384697 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:37:06.385028 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:37:06.385038 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:37:06.385049 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 02:37:06.385498 (+ 449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:06.385510 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:06.386741 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:06.386751 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:06.387936 (+ 1185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:06.387948 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:06.388760 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:06.388768 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:06.388951 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:06.389023 (+ 72us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 02:37:06.389478 (+ 455us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 02:37:06.389834 (+ 356us) client_negotiation.cc:769] Sending connection context
0812 02:37:06.389985 (+ 151us) client_negotiation.cc:240] Negotiation successful
0812 02:37:06.390151 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":561,"thread_start_us":375,"threads_started":1}
I20260812 02:37:06.428474 16998 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44856:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 02:37:06.428782 16998 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44856:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 02:37:06.432426 16998 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1c4324eb32464053b83080be530c21b3: Sending DeleteTablet for 3 replicas of tablet 7f7a41b7cd7f456db6d257ee485f2473
I20260812 02:37:06.433264 17155 tablet_service.cc:1514] Processing DeleteTablet for tablet 7f7a41b7cd7f456db6d257ee485f2473 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:37:06 UTC) from {username='oryx', principal='oryx/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:36814
I20260812 02:37:06.433470 17483 tablet_service.cc:1514] Processing DeleteTablet for tablet 7f7a41b7cd7f456db6d257ee485f2473 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:37:06 UTC) from {username='oryx', principal='oryx/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:59376
I20260812 02:37:06.438550 17671 tablet_replica.cc:330] T 7f7a41b7cd7f456db6d257ee485f2473 P e97688088d8542f7a7d708d86b197530: stopping tablet replica
I20260812 02:37:06.438794 17671 raft_consensus.cc:2229] T 7f7a41b7cd7f456db6d257ee485f2473 P e97688088d8542f7a7d708d86b197530 [term 1 LEADER]: Raft consensus shutting down.
I20260812 02:37:06.439392 17671 raft_consensus.cc:2258] T 7f7a41b7cd7f456db6d257ee485f2473 P e97688088d8542f7a7d708d86b197530 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:37:06.440055 17319 tablet_service.cc:1514] Processing DeleteTablet for tablet 7f7a41b7cd7f456db6d257ee485f2473 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:37:06 UTC) from {username='oryx', principal='oryx/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:44494
I20260812 02:37:06.447152 17671 ts_tablet_manager.cc:1905] T 7f7a41b7cd7f456db6d257ee485f2473 P e97688088d8542f7a7d708d86b197530: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:37:06.454532 17672 tablet_replica.cc:330] T 7f7a41b7cd7f456db6d257ee485f2473 P 68b44fe628d849aea0058efe05f83434: stopping tablet replica
I20260812 02:37:06.454828 17672 raft_consensus.cc:2229] T 7f7a41b7cd7f456db6d257ee485f2473 P 68b44fe628d849aea0058efe05f83434 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 02:37:06.455170 17672 raft_consensus.cc:2258] T 7f7a41b7cd7f456db6d257ee485f2473 P 68b44fe628d849aea0058efe05f83434 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:37:06.456370 17672 ts_tablet_manager.cc:1905] T 7f7a41b7cd7f456db6d257ee485f2473 P 68b44fe628d849aea0058efe05f83434: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:37:06.460620 17671 ts_tablet_manager.cc:1918] T 7f7a41b7cd7f456db6d257ee485f2473 P e97688088d8542f7a7d708d86b197530: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 02:37:06.460780 17671 log.cc:1196] T 7f7a41b7cd7f456db6d257ee485f2473 P e97688088d8542f7a7d708d86b197530: Deleting WAL directory at /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502156971023-7698-0/minicluster-data/ts-0/wal/wals/7f7a41b7cd7f456db6d257ee485f2473
I20260812 02:37:06.461248 17671 ts_tablet_manager.cc:1939] T 7f7a41b7cd7f456db6d257ee485f2473 P e97688088d8542f7a7d708d86b197530: Deleting consensus metadata
I20260812 02:37:06.462636 16975 catalog_manager.cc:4841] TS e97688088d8542f7a7d708d86b197530 (127.7.132.129:42457): tablet 7f7a41b7cd7f456db6d257ee485f2473 (table test-table [id=721127c7f06a4fcb8b8f5dabcf10f2de]) successfully deleted
I20260812 02:37:06.465576 17673 tablet_replica.cc:330] T 7f7a41b7cd7f456db6d257ee485f2473 P a0a4efea0cec47928cd91fe2ec6ce02e: stopping tablet replica
I20260812 02:37:06.465883 17673 raft_consensus.cc:2229] T 7f7a41b7cd7f456db6d257ee485f2473 P a0a4efea0cec47928cd91fe2ec6ce02e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 02:37:06.470351 17673 raft_consensus.cc:2258] T 7f7a41b7cd7f456db6d257ee485f2473 P a0a4efea0cec47928cd91fe2ec6ce02e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:37:06.471740 17673 ts_tablet_manager.cc:1905] T 7f7a41b7cd7f456db6d257ee485f2473 P a0a4efea0cec47928cd91fe2ec6ce02e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:37:06.479315 17672 ts_tablet_manager.cc:1918] T 7f7a41b7cd7f456db6d257ee485f2473 P 68b44fe628d849aea0058efe05f83434: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 02:37:06.479476 17672 log.cc:1196] T 7f7a41b7cd7f456db6d257ee485f2473 P 68b44fe628d849aea0058efe05f83434: Deleting WAL directory at /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502156971023-7698-0/minicluster-data/ts-2/wal/wals/7f7a41b7cd7f456db6d257ee485f2473
I20260812 02:37:06.479979 17672 ts_tablet_manager.cc:1939] T 7f7a41b7cd7f456db6d257ee485f2473 P 68b44fe628d849aea0058efe05f83434: Deleting consensus metadata
I20260812 02:37:06.481208 16980 catalog_manager.cc:4841] TS 68b44fe628d849aea0058efe05f83434 (127.7.132.131:33969): tablet 7f7a41b7cd7f456db6d257ee485f2473 (table test-table [id=721127c7f06a4fcb8b8f5dabcf10f2de]) successfully deleted
I20260812 02:37:06.488024 17673 ts_tablet_manager.cc:1918] T 7f7a41b7cd7f456db6d257ee485f2473 P a0a4efea0cec47928cd91fe2ec6ce02e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 02:37:06.488220 17673 log.cc:1196] T 7f7a41b7cd7f456db6d257ee485f2473 P a0a4efea0cec47928cd91fe2ec6ce02e: Deleting WAL directory at /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502156971023-7698-0/minicluster-data/ts-1/wal/wals/7f7a41b7cd7f456db6d257ee485f2473
I20260812 02:37:06.488926 17673 ts_tablet_manager.cc:1939] T 7f7a41b7cd7f456db6d257ee485f2473 P a0a4efea0cec47928cd91fe2ec6ce02e: Deleting consensus metadata
I20260812 02:37:06.490556 16975 catalog_manager.cc:4841] TS a0a4efea0cec47928cd91fe2ec6ce02e (127.7.132.130:34695): tablet 7f7a41b7cd7f456db6d257ee485f2473 (table test-table [id=721127c7f06a4fcb8b8f5dabcf10f2de]) successfully deleted
Aug 12 02:37:06 dist-test-slave-mn4r krb5kdc[16940](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502226, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 02:37:06 dist-test-slave-mn4r krb5kdc[16940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 02:37:06 dist-test-slave-mn4r krb5kdc[16940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
W20260812 02:37:06.528700 17433 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:37:06.516860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36150 (local address 127.7.132.190:38167)
0812 02:37:06.517083 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:06.517092 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:06.517672 (+ 580us) server_negotiation.cc:407] Connection header received
0812 02:37:06.517895 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:06.517902 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:06.518027 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:06.518171 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:37:06.519844 (+ 1673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:06.520662 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:06.522163 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:06.522656 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:06.522800 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:06.528441 (+ 5641us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36150: BlockingRecv error: recv got EOF from 127.0.0.1:36150 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
Aug 12 02:37:06 dist-test-slave-mn4r krb5kdc[16940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502226, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM
I20260812 02:37:06.548130 17433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:06.532395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36156 (local address 127.7.132.190:38167)
0812 02:37:06.532623 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:06.532632 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:06.534602 (+ 1970us) server_negotiation.cc:407] Connection header received
0812 02:37:06.534797 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:06.534802 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:06.534874 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:06.534977 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:37:06.536119 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:06.536849 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:06.538122 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:06.538568 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:06.538700 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:06.542649 (+ 3949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:37:06.542658 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:37:06.542662 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:37:06.542701 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:37:06.545370 (+ 2669us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:37:06.546056 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:37:06.546061 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:37:06.546063 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:37:06.546247 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:37:06.546642 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:37:06.546648 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:37:06.546651 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:37:06.546909 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:37:06.547182 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0812 02:37:06.547466 (+ 284us) server_negotiation.cc:299] Negotiation successful
0812 02:37:06.547687 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260812 02:37:06.551618 16998 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36156:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 02:37:06.552263 16998 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 02:37:06.559300 17155 tablet_service.cc:1467] Processing CreateTablet for tablet 26efc4e0ac074348be202863792acf90 (DEFAULT_TABLE table=test-table [id=e6aec041a2174e1f91b3cb2553bfc5be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:37:06.559760 17155 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26efc4e0ac074348be202863792acf90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:37:06.559994 17483 tablet_service.cc:1467] Processing CreateTablet for tablet 26efc4e0ac074348be202863792acf90 (DEFAULT_TABLE table=test-table [id=e6aec041a2174e1f91b3cb2553bfc5be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:37:06.560482 17483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26efc4e0ac074348be202863792acf90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:37:06.562425 17319 tablet_service.cc:1467] Processing CreateTablet for tablet 26efc4e0ac074348be202863792acf90 (DEFAULT_TABLE table=test-table [id=e6aec041a2174e1f91b3cb2553bfc5be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:37:06.562842 17319 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26efc4e0ac074348be202863792acf90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:37:06.567462 17592 tablet_bootstrap.cc:492] T 26efc4e0ac074348be202863792acf90 P 68b44fe628d849aea0058efe05f83434: Bootstrap starting.
I20260812 02:37:06.569229 17592 tablet_bootstrap.cc:654] T 26efc4e0ac074348be202863792acf90 P 68b44fe628d849aea0058efe05f83434: Neither blocks nor log segments found. Creating new log.
I20260812 02:37:06.573287 17593 tablet_bootstrap.cc:492] T 26efc4e0ac074348be202863792acf90 P a0a4efea0cec47928cd91fe2ec6ce02e: Bootstrap starting.
I20260812 02:37:06.573736 17591 tablet_bootstrap.cc:492] T 26efc4e0ac074348be202863792acf90 P e97688088d8542f7a7d708d86b197530: Bootstrap starting.
I20260812 02:37:06.575027 17593 tablet_bootstrap.cc:654] T 26efc4e0ac074348be202863792acf90 P a0a4efea0cec47928cd91fe2ec6ce02e: Neither blocks nor log segments found. Creating new log.
I20260812 02:37:06.575346 17591 tablet_bootstrap.cc:654] T 26efc4e0ac074348be202863792acf90 P e97688088d8542f7a7d708d86b197530: Neither blocks nor log segments found. Creating new log.
I20260812 02:37:06.578866 17592 tablet_bootstrap.cc:492] T 26efc4e0ac074348be202863792acf90 P 68b44fe628d849aea0058efe05f83434: No bootstrap required, opened a new log
I20260812 02:37:06.579052 17592 ts_tablet_manager.cc:1397] T 26efc4e0ac074348be202863792acf90 P 68b44fe628d849aea0058efe05f83434: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.000s
I20260812 02:37:06.579856 17592 raft_consensus.cc:348] T 26efc4e0ac074348be202863792acf90 P 68b44fe628d849aea0058efe05f83434 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e97688088d8542f7a7d708d86b197530" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 42457 } } peers { permanent_uuid: "a0a4efea0cec47928cd91fe2ec6ce02e" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34695 } } peers { permanent_uuid: "68b44fe628d849aea0058efe05f83434" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33969 } }
I20260812 02:37:06.580173 17592 raft_consensus.cc:374] T 26efc4e0ac074348be202863792acf90 P 68b44fe628d849aea0058efe05f83434 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:37:06.580268 17592 raft_consensus.cc:729] T 26efc4e0ac074348be202863792acf90 P 68b44fe628d849aea0058efe05f83434 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68b44fe628d849aea0058efe05f83434, State: Initialized, Role: FOLLOWER
I20260812 02:37:06.580508 17592 consensus_queue.cc:260] T 26efc4e0ac074348be202863792acf90 P 68b44fe628d849aea0058efe05f83434 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e97688088d8542f7a7d708d86b197530" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 42457 } } peers { permanent_uuid: "a0a4efea0cec47928cd91fe2ec6ce02e" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34695 } } peers { permanent_uuid: "68b44fe628d849aea0058efe05f83434" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33969 } }
I20260812 02:37:06.581202 17592 ts_tablet_manager.cc:1428] T 26efc4e0ac074348be202863792acf90 P 68b44fe628d849aea0058efe05f83434: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260812 02:37:06.587976 17591 tablet_bootstrap.cc:492] T 26efc4e0ac074348be202863792acf90 P e97688088d8542f7a7d708d86b197530: No bootstrap required, opened a new log
I20260812 02:37:06.588167 17591 ts_tablet_manager.cc:1397] T 26efc4e0ac074348be202863792acf90 P e97688088d8542f7a7d708d86b197530: Time spent bootstrapping tablet: real 0.015s user 0.006s sys 0.000s
I20260812 02:37:06.589080 17591 raft_consensus.cc:348] T 26efc4e0ac074348be202863792acf90 P e97688088d8542f7a7d708d86b197530 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e97688088d8542f7a7d708d86b197530" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 42457 } } peers { permanent_uuid: "a0a4efea0cec47928cd91fe2ec6ce02e" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34695 } } peers { permanent_uuid: "68b44fe628d849aea0058efe05f83434" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33969 } }
I20260812 02:37:06.589381 17591 raft_consensus.cc:374] T 26efc4e0ac074348be202863792acf90 P e97688088d8542f7a7d708d86b197530 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:37:06.589502 17591 raft_consensus.cc:729] T 26efc4e0ac074348be202863792acf90 P e97688088d8542f7a7d708d86b197530 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e97688088d8542f7a7d708d86b197530, State: Initialized, Role: FOLLOWER
I20260812 02:37:06.589799 17591 consensus_queue.cc:260] T 26efc4e0ac074348be202863792acf90 P e97688088d8542f7a7d708d86b197530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e97688088d8542f7a7d708d86b197530" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 42457 } } peers { permanent_uuid: "a0a4efea0cec47928cd91fe2ec6ce02e" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34695 } } peers { permanent_uuid: "68b44fe628d849aea0058efe05f83434" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33969 } }
I20260812 02:37:06.592797 17591 ts_tablet_manager.cc:1428] T 26efc4e0ac074348be202863792acf90 P e97688088d8542f7a7d708d86b197530: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260812 02:37:06.598527 17593 tablet_bootstrap.cc:492] T 26efc4e0ac074348be202863792acf90 P a0a4efea0cec47928cd91fe2ec6ce02e: No bootstrap required, opened a new log
I20260812 02:37:06.598701 17593 ts_tablet_manager.cc:1397] T 26efc4e0ac074348be202863792acf90 P a0a4efea0cec47928cd91fe2ec6ce02e: Time spent bootstrapping tablet: real 0.026s user 0.004s sys 0.011s
I20260812 02:37:06.599645 17593 raft_consensus.cc:348] T 26efc4e0ac074348be202863792acf90 P a0a4efea0cec47928cd91fe2ec6ce02e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e97688088d8542f7a7d708d86b197530" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 42457 } } peers { permanent_uuid: "a0a4efea0cec47928cd91fe2ec6ce02e" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34695 } } peers { permanent_uuid: "68b44fe628d849aea0058efe05f83434" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33969 } }
I20260812 02:37:06.599936 17593 raft_consensus.cc:374] T 26efc4e0ac074348be202863792acf90 P a0a4efea0cec47928cd91fe2ec6ce02e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:37:06.600032 17593 raft_consensus.cc:729] T 26efc4e0ac074348be202863792acf90 P a0a4efea0cec47928cd91fe2ec6ce02e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0a4efea0cec47928cd91fe2ec6ce02e, State: Initialized, Role: FOLLOWER
I20260812 02:37:06.600308 17593 consensus_queue.cc:260] T 26efc4e0ac074348be202863792acf90 P a0a4efea0cec47928cd91fe2ec6ce02e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e97688088d8542f7a7d708d86b197530" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 42457 } } peers { permanent_uuid: "a0a4efea0cec47928cd91fe2ec6ce02e" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34695 } } peers { permanent_uuid: "68b44fe628d849aea0058efe05f83434" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33969 } }
I20260812 02:37:06.603292 17593 ts_tablet_manager.cc:1428] T 26efc4e0ac074348be202863792acf90 P a0a4efea0cec47928cd91fe2ec6ce02e: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20260812 02:37:06.614969 17599 raft_consensus.cc:482] T 26efc4e0ac074348be202863792acf90 P 68b44fe628d849aea0058efe05f83434 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 02:37:06.615167 17599 raft_consensus.cc:504] T 26efc4e0ac074348be202863792acf90 P 68b44fe628d849aea0058efe05f83434 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e97688088d8542f7a7d708d86b197530" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 42457 } } peers { permanent_uuid: "a0a4efea0cec47928cd91fe2ec6ce02e" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34695 } } peers { permanent_uuid: "68b44fe628d849aea0058efe05f83434" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33969 } }
I20260812 02:37:06.615862 17599 leader_election.cc:288] T 26efc4e0ac074348be202863792acf90 P 68b44fe628d849aea0058efe05f83434 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e97688088d8542f7a7d708d86b197530 (127.7.132.129:42457), a0a4efea0cec47928cd91fe2ec6ce02e (127.7.132.130:34695)
I20260812 02:37:06.616791 17180 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26efc4e0ac074348be202863792acf90" candidate_uuid: "68b44fe628d849aea0058efe05f83434" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e97688088d8542f7a7d708d86b197530" is_pre_election: true
I20260812 02:37:06.617110 17180 raft_consensus.cc:2454] T 26efc4e0ac074348be202863792acf90 P e97688088d8542f7a7d708d86b197530 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68b44fe628d849aea0058efe05f83434 in term 0.
I20260812 02:37:06.617074 17349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26efc4e0ac074348be202863792acf90" candidate_uuid: "68b44fe628d849aea0058efe05f83434" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0a4efea0cec47928cd91fe2ec6ce02e" is_pre_election: true
I20260812 02:37:06.617353 17349 raft_consensus.cc:2454] T 26efc4e0ac074348be202863792acf90 P a0a4efea0cec47928cd91fe2ec6ce02e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68b44fe628d849aea0058efe05f83434 in term 0.
I20260812 02:37:06.617723 17425 leader_election.cc:302] T 26efc4e0ac074348be202863792acf90 P 68b44fe628d849aea0058efe05f83434 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68b44fe628d849aea0058efe05f83434, e97688088d8542f7a7d708d86b197530; no voters:
I20260812 02:37:06.618023 17599 raft_consensus.cc:2781] T 26efc4e0ac074348be202863792acf90 P 68b44fe628d849aea0058efe05f83434 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 02:37:06.618175 17599 raft_consensus.cc:482] T 26efc4e0ac074348be202863792acf90 P 68b44fe628d849aea0058efe05f83434 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 02:37:06.618289 17599 raft_consensus.cc:3037] T 26efc4e0ac074348be202863792acf90 P 68b44fe628d849aea0058efe05f83434 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:37:06.620606 17599 raft_consensus.cc:504] T 26efc4e0ac074348be202863792acf90 P 68b44fe628d849aea0058efe05f83434 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e97688088d8542f7a7d708d86b197530" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 42457 } } peers { permanent_uuid: "a0a4efea0cec47928cd91fe2ec6ce02e" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34695 } } peers { permanent_uuid: "68b44fe628d849aea0058efe05f83434" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33969 } }
I20260812 02:37:06.621157 17599 leader_election.cc:288] T 26efc4e0ac074348be202863792acf90 P 68b44fe628d849aea0058efe05f83434 [CANDIDATE]: Term 1 election: Requested vote from peers e97688088d8542f7a7d708d86b197530 (127.7.132.129:42457), a0a4efea0cec47928cd91fe2ec6ce02e (127.7.132.130:34695)
I20260812 02:37:06.621722 17349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26efc4e0ac074348be202863792acf90" candidate_uuid: "68b44fe628d849aea0058efe05f83434" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0a4efea0cec47928cd91fe2ec6ce02e"
I20260812 02:37:06.622061 17349 raft_consensus.cc:3037] T 26efc4e0ac074348be202863792acf90 P a0a4efea0cec47928cd91fe2ec6ce02e [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:37:06.625454 17349 raft_consensus.cc:2454] T 26efc4e0ac074348be202863792acf90 P a0a4efea0cec47928cd91fe2ec6ce02e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68b44fe628d849aea0058efe05f83434 in term 1.
I20260812 02:37:06.625936 17425 leader_election.cc:302] T 26efc4e0ac074348be202863792acf90 P 68b44fe628d849aea0058efe05f83434 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68b44fe628d849aea0058efe05f83434, a0a4efea0cec47928cd91fe2ec6ce02e; no voters:
I20260812 02:37:06.626288 17599 raft_consensus.cc:2781] T 26efc4e0ac074348be202863792acf90 P 68b44fe628d849aea0058efe05f83434 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:37:06.627465 17180 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26efc4e0ac074348be202863792acf90" candidate_uuid: "68b44fe628d849aea0058efe05f83434" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e97688088d8542f7a7d708d86b197530"
I20260812 02:37:06.627682 17180 raft_consensus.cc:3037] T 26efc4e0ac074348be202863792acf90 P e97688088d8542f7a7d708d86b197530 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:37:06.631319 17180 raft_consensus.cc:2454] T 26efc4e0ac074348be202863792acf90 P e97688088d8542f7a7d708d86b197530 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68b44fe628d849aea0058efe05f83434 in term 1.
I20260812 02:37:06.632206 17599 raft_consensus.cc:686] T 26efc4e0ac074348be202863792acf90 P 68b44fe628d849aea0058efe05f83434 [term 1 LEADER]: Becoming Leader. State: Replica: 68b44fe628d849aea0058efe05f83434, State: Running, Role: LEADER
I20260812 02:37:06.632570 17599 consensus_queue.cc:237] T 26efc4e0ac074348be202863792acf90 P 68b44fe628d849aea0058efe05f83434 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e97688088d8542f7a7d708d86b197530" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 42457 } } peers { permanent_uuid: "a0a4efea0cec47928cd91fe2ec6ce02e" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34695 } } peers { permanent_uuid: "68b44fe628d849aea0058efe05f83434" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33969 } }
I20260812 02:37:06.636204 16997 catalog_manager.cc:5495] T 26efc4e0ac074348be202863792acf90 P 68b44fe628d849aea0058efe05f83434 reported cstate change: term changed from 0 to 1, leader changed from <none> to 68b44fe628d849aea0058efe05f83434 (127.7.132.131). New cstate: current_term: 1 leader_uuid: "68b44fe628d849aea0058efe05f83434" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e97688088d8542f7a7d708d86b197530" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 42457 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0a4efea0cec47928cd91fe2ec6ce02e" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34695 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "68b44fe628d849aea0058efe05f83434" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33969 } health_report { overall_health: HEALTHY } } }
I20260812 02:37:06.652894 17588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:06.646301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55846 (local address 127.7.132.131:33969)
0812 02:37:06.646496 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:06.646502 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:06.646524 (+ 22us) server_negotiation.cc:407] Connection header received
0812 02:37:06.646595 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:06.646601 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:06.646691 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:06.646801 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 02:37:06.647266 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:06.648370 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:06.650553 (+ 2183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:06.651266 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:06.651390 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:06.652056 (+ 666us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 02:37:06.652222 (+ 166us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 02:37:06.652324 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0812 02:37:06.652486 (+ 162us) server_negotiation.cc:299] Negotiation successful
0812 02:37:06.652580 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260812 02:37:06.656370 17349 raft_consensus.cc:1261] T 26efc4e0ac074348be202863792acf90 P a0a4efea0cec47928cd91fe2ec6ce02e [term 1 FOLLOWER]: Refusing update from remote peer 68b44fe628d849aea0058efe05f83434: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:37:06.656546 17180 raft_consensus.cc:1261] T 26efc4e0ac074348be202863792acf90 P e97688088d8542f7a7d708d86b197530 [term 1 FOLLOWER]: Refusing update from remote peer 68b44fe628d849aea0058efe05f83434: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:37:06.657243 17599 consensus_queue.cc:1035] T 26efc4e0ac074348be202863792acf90 P 68b44fe628d849aea0058efe05f83434 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e97688088d8542f7a7d708d86b197530" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 42457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:37:06.657640 17693 consensus_queue.cc:1035] T 26efc4e0ac074348be202863792acf90 P 68b44fe628d849aea0058efe05f83434 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0a4efea0cec47928cd91fe2ec6ce02e" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:37:06.662635 17665 mvcc.cc:205] Tried to move back new op lower bound from 7317513120377008128 to 7317513120290103296. Current Snapshot: MvccSnapshot[applied={T|T < 7317513120377008128}]
I20260812 02:37:06.670542 16997 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36156:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 02:37:06.670822 16997 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36156:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 02:37:06.675169 16997 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1c4324eb32464053b83080be530c21b3: Sending DeleteTablet for 3 replicas of tablet 26efc4e0ac074348be202863792acf90
I20260812 02:37:06.676429 17155 tablet_service.cc:1514] Processing DeleteTablet for tablet 26efc4e0ac074348be202863792acf90 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:37:06 UTC) from {username='oryx', principal='oryx/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:36814
I20260812 02:37:06.676792 17671 tablet_replica.cc:330] T 26efc4e0ac074348be202863792acf90 P e97688088d8542f7a7d708d86b197530: stopping tablet replica
I20260812 02:37:06.676811 17319 tablet_service.cc:1514] Processing DeleteTablet for tablet 26efc4e0ac074348be202863792acf90 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:37:06 UTC) from {username='oryx', principal='oryx/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:44494
I20260812 02:37:06.676893 17483 tablet_service.cc:1514] Processing DeleteTablet for tablet 26efc4e0ac074348be202863792acf90 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:37:06 UTC) from {username='oryx', principal='oryx/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:59376
I20260812 02:37:06.677007 17671 raft_consensus.cc:2229] T 26efc4e0ac074348be202863792acf90 P e97688088d8542f7a7d708d86b197530 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 02:37:06.677111 17673 tablet_replica.cc:330] T 26efc4e0ac074348be202863792acf90 P a0a4efea0cec47928cd91fe2ec6ce02e: stopping tablet replica
I20260812 02:37:06.677238 17671 raft_consensus.cc:2258] T 26efc4e0ac074348be202863792acf90 P e97688088d8542f7a7d708d86b197530 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:37:06.677503 17672 tablet_replica.cc:330] T 26efc4e0ac074348be202863792acf90 P 68b44fe628d849aea0058efe05f83434: stopping tablet replica
I20260812 02:37:06.677692 17672 raft_consensus.cc:2229] T 26efc4e0ac074348be202863792acf90 P 68b44fe628d849aea0058efe05f83434 [term 1 LEADER]: Raft consensus shutting down.
I20260812 02:37:06.678224 17672 raft_consensus.cc:2258] T 26efc4e0ac074348be202863792acf90 P 68b44fe628d849aea0058efe05f83434 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:37:06.678791 16997 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36156:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 02:37:06.679509 16997 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 02:37:06.679876 17673 raft_consensus.cc:2229] T 26efc4e0ac074348be202863792acf90 P a0a4efea0cec47928cd91fe2ec6ce02e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 02:37:06.680449 17673 pending_rounds.cc:62] T 26efc4e0ac074348be202863792acf90 P a0a4efea0cec47928cd91fe2ec6ce02e: Trying to abort 2 pending ops.
I20260812 02:37:06.680596 17673 pending_rounds.cc:68] T 26efc4e0ac074348be202863792acf90 P a0a4efea0cec47928cd91fe2ec6ce02e: Aborting op as it isn't in flight: id { term: 1 index: 2 } timestamp: 7317513120377008128 op_type: WRITE_OP write_request { tablet_id: "26efc4e0ac074348be202863792acf90" schema { columns { name: "key" type: INT32 is_key: true is_nullable: false immutable: false } columns { name: "val" type: INT32 is_key: false is_nullable: true immutable: false } } row_operations { rows: "<redacted>""\001\003\00090\000\0001\324\000\000" indirect_data: "<redacted>""" } external_consistency_mode: CLIENT_PROPAGATED authz_token { token_data: "\010\376\272\357\323\006\"8\n\ntest-admin\022*\n e6aec041a2174e1f91b3cb2553bfc5be\020\001\030\001 \001(\001" signature: "<redacted>""\353\2100v!\333\314W\272<\370\354|\036\367\327\246\2772\017PV8CO\323Mpf\007~\006\305;_C\014\211\010\376\320\376\374\376\3262\237\250\255\264\354\020S\316\302\036:\222\207;\370\337,\214" signing_key_seq_num: 0 } } request_id { client_id: "85bf210e89504ea38d091eb6d70b7f9a" seq_no: 0 first_incomplete_seq_no: 0 attempt_no: 0 }
I20260812 02:37:06.681396 17673 pending_rounds.cc:68] T 26efc4e0ac074348be202863792acf90 P a0a4efea0cec47928cd91fe2ec6ce02e: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7317513120290103296 op_type: NO_OP noop_request { }
I20260812 02:37:06.681594 17673 raft_consensus.cc:2866] T 26efc4e0ac074348be202863792acf90 P a0a4efea0cec47928cd91fe2ec6ce02e [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260812 02:37:06.681726 17673 raft_consensus.cc:2258] T 26efc4e0ac074348be202863792acf90 P a0a4efea0cec47928cd91fe2ec6ce02e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:37:06.683542 17671 ts_tablet_manager.cc:1905] T 26efc4e0ac074348be202863792acf90 P e97688088d8542f7a7d708d86b197530: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:37:06.688301 17672 ts_tablet_manager.cc:1905] T 26efc4e0ac074348be202863792acf90 P 68b44fe628d849aea0058efe05f83434: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:37:06.691033 17673 ts_tablet_manager.cc:1905] T 26efc4e0ac074348be202863792acf90 P a0a4efea0cec47928cd91fe2ec6ce02e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:37:06.691596 17671 ts_tablet_manager.cc:1918] T 26efc4e0ac074348be202863792acf90 P e97688088d8542f7a7d708d86b197530: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 02:37:06.691807 17671 log.cc:1196] T 26efc4e0ac074348be202863792acf90 P e97688088d8542f7a7d708d86b197530: Deleting WAL directory at /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502156971023-7698-0/minicluster-data/ts-0/wal/wals/26efc4e0ac074348be202863792acf90
I20260812 02:37:06.692346 17671 ts_tablet_manager.cc:1939] T 26efc4e0ac074348be202863792acf90 P e97688088d8542f7a7d708d86b197530: Deleting consensus metadata
I20260812 02:37:06.693864 16975 catalog_manager.cc:4841] TS e97688088d8542f7a7d708d86b197530 (127.7.132.129:42457): tablet 26efc4e0ac074348be202863792acf90 (table test-table [id=e6aec041a2174e1f91b3cb2553bfc5be]) successfully deleted
I20260812 02:37:06.697865 17673 ts_tablet_manager.cc:1918] T 26efc4e0ac074348be202863792acf90 P a0a4efea0cec47928cd91fe2ec6ce02e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 02:37:06.698024 17673 log.cc:1196] T 26efc4e0ac074348be202863792acf90 P a0a4efea0cec47928cd91fe2ec6ce02e: Deleting WAL directory at /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502156971023-7698-0/minicluster-data/ts-1/wal/wals/26efc4e0ac074348be202863792acf90
I20260812 02:37:06.698495 17673 ts_tablet_manager.cc:1939] T 26efc4e0ac074348be202863792acf90 P a0a4efea0cec47928cd91fe2ec6ce02e: Deleting consensus metadata
I20260812 02:37:06.698880 17319 tablet_service.cc:1467] Processing CreateTablet for tablet 4793c0463cd4487596b1f6c3ba8b4ed6 (DEFAULT_TABLE table=test-table [id=2a7f607ffe3647e797b210078913a6cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:37:06.699322 17319 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4793c0463cd4487596b1f6c3ba8b4ed6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:37:06.702273 17672 ts_tablet_manager.cc:1918] T 26efc4e0ac074348be202863792acf90 P 68b44fe628d849aea0058efe05f83434: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 02:37:06.702513 17672 log.cc:1196] T 26efc4e0ac074348be202863792acf90 P 68b44fe628d849aea0058efe05f83434: Deleting WAL directory at /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502156971023-7698-0/minicluster-data/ts-2/wal/wals/26efc4e0ac074348be202863792acf90
I20260812 02:37:06.703101 17672 ts_tablet_manager.cc:1939] T 26efc4e0ac074348be202863792acf90 P 68b44fe628d849aea0058efe05f83434: Deleting consensus metadata
I20260812 02:37:06.704159 16975 catalog_manager.cc:4841] TS a0a4efea0cec47928cd91fe2ec6ce02e (127.7.132.130:34695): tablet 26efc4e0ac074348be202863792acf90 (table test-table [id=e6aec041a2174e1f91b3cb2553bfc5be]) successfully deleted
I20260812 02:37:06.705572 17483 tablet_service.cc:1467] Processing CreateTablet for tablet 4793c0463cd4487596b1f6c3ba8b4ed6 (DEFAULT_TABLE table=test-table [id=2a7f607ffe3647e797b210078913a6cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:37:06.705994 17483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4793c0463cd4487596b1f6c3ba8b4ed6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:37:06.706661 16980 catalog_manager.cc:4841] TS 68b44fe628d849aea0058efe05f83434 (127.7.132.131:33969): tablet 26efc4e0ac074348be202863792acf90 (table test-table [id=e6aec041a2174e1f91b3cb2553bfc5be]) successfully deleted
I20260812 02:37:06.706884 17155 tablet_service.cc:1467] Processing CreateTablet for tablet 4793c0463cd4487596b1f6c3ba8b4ed6 (DEFAULT_TABLE table=test-table [id=2a7f607ffe3647e797b210078913a6cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:37:06.707319 17155 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4793c0463cd4487596b1f6c3ba8b4ed6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:37:06.714007 17591 tablet_bootstrap.cc:492] T 4793c0463cd4487596b1f6c3ba8b4ed6 P e97688088d8542f7a7d708d86b197530: Bootstrap starting.
I20260812 02:37:06.715796 17591 tablet_bootstrap.cc:654] T 4793c0463cd4487596b1f6c3ba8b4ed6 P e97688088d8542f7a7d708d86b197530: Neither blocks nor log segments found. Creating new log.
I20260812 02:37:06.716001 17592 tablet_bootstrap.cc:492] T 4793c0463cd4487596b1f6c3ba8b4ed6 P 68b44fe628d849aea0058efe05f83434: Bootstrap starting.
I20260812 02:37:06.717746 17592 tablet_bootstrap.cc:654] T 4793c0463cd4487596b1f6c3ba8b4ed6 P 68b44fe628d849aea0058efe05f83434: Neither blocks nor log segments found. Creating new log.
I20260812 02:37:06.720012 17593 tablet_bootstrap.cc:492] T 4793c0463cd4487596b1f6c3ba8b4ed6 P a0a4efea0cec47928cd91fe2ec6ce02e: Bootstrap starting.
I20260812 02:37:06.721977 17593 tablet_bootstrap.cc:654] T 4793c0463cd4487596b1f6c3ba8b4ed6 P a0a4efea0cec47928cd91fe2ec6ce02e: Neither blocks nor log segments found. Creating new log.
I20260812 02:37:06.727356 17591 tablet_bootstrap.cc:492] T 4793c0463cd4487596b1f6c3ba8b4ed6 P e97688088d8542f7a7d708d86b197530: No bootstrap required, opened a new log
I20260812 02:37:06.727581 17591 ts_tablet_manager.cc:1397] T 4793c0463cd4487596b1f6c3ba8b4ed6 P e97688088d8542f7a7d708d86b197530: Time spent bootstrapping tablet: real 0.014s user 0.004s sys 0.006s
I20260812 02:37:06.728935 17591 raft_consensus.cc:348] T 4793c0463cd4487596b1f6c3ba8b4ed6 P e97688088d8542f7a7d708d86b197530 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0a4efea0cec47928cd91fe2ec6ce02e" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34695 } } peers { permanent_uuid: "e97688088d8542f7a7d708d86b197530" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 42457 } } peers { permanent_uuid: "68b44fe628d849aea0058efe05f83434" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33969 } }
I20260812 02:37:06.729292 17591 raft_consensus.cc:374] T 4793c0463cd4487596b1f6c3ba8b4ed6 P e97688088d8542f7a7d708d86b197530 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:37:06.729375 17591 raft_consensus.cc:729] T 4793c0463cd4487596b1f6c3ba8b4ed6 P e97688088d8542f7a7d708d86b197530 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e97688088d8542f7a7d708d86b197530, State: Initialized, Role: FOLLOWER
I20260812 02:37:06.729842 17591 consensus_queue.cc:260] T 4793c0463cd4487596b1f6c3ba8b4ed6 P e97688088d8542f7a7d708d86b197530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0a4efea0cec47928cd91fe2ec6ce02e" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34695 } } peers { permanent_uuid: "e97688088d8542f7a7d708d86b197530" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 42457 } } peers { permanent_uuid: "68b44fe628d849aea0058efe05f83434" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33969 } }
I20260812 02:37:06.731303 17591 ts_tablet_manager.cc:1428] T 4793c0463cd4487596b1f6c3ba8b4ed6 P e97688088d8542f7a7d708d86b197530: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260812 02:37:06.731755 17593 tablet_bootstrap.cc:492] T 4793c0463cd4487596b1f6c3ba8b4ed6 P a0a4efea0cec47928cd91fe2ec6ce02e: No bootstrap required, opened a new log
I20260812 02:37:06.731945 17593 ts_tablet_manager.cc:1397] T 4793c0463cd4487596b1f6c3ba8b4ed6 P a0a4efea0cec47928cd91fe2ec6ce02e: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.002s
I20260812 02:37:06.732676 17593 raft_consensus.cc:348] T 4793c0463cd4487596b1f6c3ba8b4ed6 P a0a4efea0cec47928cd91fe2ec6ce02e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0a4efea0cec47928cd91fe2ec6ce02e" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34695 } } peers { permanent_uuid: "e97688088d8542f7a7d708d86b197530" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 42457 } } peers { permanent_uuid: "68b44fe628d849aea0058efe05f83434" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33969 } }
I20260812 02:37:06.732937 17593 raft_consensus.cc:374] T 4793c0463cd4487596b1f6c3ba8b4ed6 P a0a4efea0cec47928cd91fe2ec6ce02e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:37:06.733036 17593 raft_consensus.cc:729] T 4793c0463cd4487596b1f6c3ba8b4ed6 P a0a4efea0cec47928cd91fe2ec6ce02e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0a4efea0cec47928cd91fe2ec6ce02e, State: Initialized, Role: FOLLOWER
I20260812 02:37:06.733359 17593 consensus_queue.cc:260] T 4793c0463cd4487596b1f6c3ba8b4ed6 P a0a4efea0cec47928cd91fe2ec6ce02e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0a4efea0cec47928cd91fe2ec6ce02e" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34695 } } peers { permanent_uuid: "e97688088d8542f7a7d708d86b197530" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 42457 } } peers { permanent_uuid: "68b44fe628d849aea0058efe05f83434" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33969 } }
I20260812 02:37:06.734000 17593 ts_tablet_manager.cc:1428] T 4793c0463cd4487596b1f6c3ba8b4ed6 P a0a4efea0cec47928cd91fe2ec6ce02e: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260812 02:37:06.739409 17592 tablet_bootstrap.cc:492] T 4793c0463cd4487596b1f6c3ba8b4ed6 P 68b44fe628d849aea0058efe05f83434: No bootstrap required, opened a new log
I20260812 02:37:06.739594 17592 ts_tablet_manager.cc:1397] T 4793c0463cd4487596b1f6c3ba8b4ed6 P 68b44fe628d849aea0058efe05f83434: Time spent bootstrapping tablet: real 0.024s user 0.006s sys 0.004s
I20260812 02:37:06.740391 17592 raft_consensus.cc:348] T 4793c0463cd4487596b1f6c3ba8b4ed6 P 68b44fe628d849aea0058efe05f83434 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0a4efea0cec47928cd91fe2ec6ce02e" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34695 } } peers { permanent_uuid: "e97688088d8542f7a7d708d86b197530" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 42457 } } peers { permanent_uuid: "68b44fe628d849aea0058efe05f83434" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33969 } }
I20260812 02:37:06.740651 17592 raft_consensus.cc:374] T 4793c0463cd4487596b1f6c3ba8b4ed6 P 68b44fe628d849aea0058efe05f83434 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:37:06.740733 17592 raft_consensus.cc:729] T 4793c0463cd4487596b1f6c3ba8b4ed6 P 68b44fe628d849aea0058efe05f83434 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68b44fe628d849aea0058efe05f83434, State: Initialized, Role: FOLLOWER
I20260812 02:37:06.740978 17592 consensus_queue.cc:260] T 4793c0463cd4487596b1f6c3ba8b4ed6 P 68b44fe628d849aea0058efe05f83434 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0a4efea0cec47928cd91fe2ec6ce02e" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34695 } } peers { permanent_uuid: "e97688088d8542f7a7d708d86b197530" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 42457 } } peers { permanent_uuid: "68b44fe628d849aea0058efe05f83434" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33969 } }
I20260812 02:37:06.741525 17592 ts_tablet_manager.cc:1428] T 4793c0463cd4487596b1f6c3ba8b4ed6 P 68b44fe628d849aea0058efe05f83434: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 02:37:07.009573 17664 raft_consensus.cc:482] T 4793c0463cd4487596b1f6c3ba8b4ed6 P e97688088d8542f7a7d708d86b197530 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 02:37:07.009781 17664 raft_consensus.cc:504] T 4793c0463cd4487596b1f6c3ba8b4ed6 P e97688088d8542f7a7d708d86b197530 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0a4efea0cec47928cd91fe2ec6ce02e" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34695 } } peers { permanent_uuid: "e97688088d8542f7a7d708d86b197530" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 42457 } } peers { permanent_uuid: "68b44fe628d849aea0058efe05f83434" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33969 } }
I20260812 02:37:07.010584 17664 leader_election.cc:288] T 4793c0463cd4487596b1f6c3ba8b4ed6 P e97688088d8542f7a7d708d86b197530 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0a4efea0cec47928cd91fe2ec6ce02e (127.7.132.130:34695), 68b44fe628d849aea0058efe05f83434 (127.7.132.131:33969)
I20260812 02:37:07.011138 17349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4793c0463cd4487596b1f6c3ba8b4ed6" candidate_uuid: "e97688088d8542f7a7d708d86b197530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0a4efea0cec47928cd91fe2ec6ce02e" is_pre_election: true
I20260812 02:37:07.011166 17510 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4793c0463cd4487596b1f6c3ba8b4ed6" candidate_uuid: "e97688088d8542f7a7d708d86b197530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68b44fe628d849aea0058efe05f83434" is_pre_election: true
I20260812 02:37:07.011370 17349 raft_consensus.cc:2454] T 4793c0463cd4487596b1f6c3ba8b4ed6 P a0a4efea0cec47928cd91fe2ec6ce02e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e97688088d8542f7a7d708d86b197530 in term 0.
I20260812 02:37:07.011456 17510 raft_consensus.cc:2454] T 4793c0463cd4487596b1f6c3ba8b4ed6 P 68b44fe628d849aea0058efe05f83434 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e97688088d8542f7a7d708d86b197530 in term 0.
I20260812 02:37:07.011881 17098 leader_election.cc:302] T 4793c0463cd4487596b1f6c3ba8b4ed6 P e97688088d8542f7a7d708d86b197530 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a0a4efea0cec47928cd91fe2ec6ce02e, e97688088d8542f7a7d708d86b197530; no voters:
I20260812 02:37:07.012233 17664 raft_consensus.cc:2781] T 4793c0463cd4487596b1f6c3ba8b4ed6 P e97688088d8542f7a7d708d86b197530 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 02:37:07.012370 17664 raft_consensus.cc:482] T 4793c0463cd4487596b1f6c3ba8b4ed6 P e97688088d8542f7a7d708d86b197530 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 02:37:07.012483 17664 raft_consensus.cc:3037] T 4793c0463cd4487596b1f6c3ba8b4ed6 P e97688088d8542f7a7d708d86b197530 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:37:07.014678 17664 raft_consensus.cc:504] T 4793c0463cd4487596b1f6c3ba8b4ed6 P e97688088d8542f7a7d708d86b197530 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0a4efea0cec47928cd91fe2ec6ce02e" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34695 } } peers { permanent_uuid: "e97688088d8542f7a7d708d86b197530" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 42457 } } peers { permanent_uuid: "68b44fe628d849aea0058efe05f83434" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33969 } }
I20260812 02:37:07.015225 17664 leader_election.cc:288] T 4793c0463cd4487596b1f6c3ba8b4ed6 P e97688088d8542f7a7d708d86b197530 [CANDIDATE]: Term 1 election: Requested vote from peers a0a4efea0cec47928cd91fe2ec6ce02e (127.7.132.130:34695), 68b44fe628d849aea0058efe05f83434 (127.7.132.131:33969)
I20260812 02:37:07.015766 17510 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4793c0463cd4487596b1f6c3ba8b4ed6" candidate_uuid: "e97688088d8542f7a7d708d86b197530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68b44fe628d849aea0058efe05f83434"
I20260812 02:37:07.015746 17349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4793c0463cd4487596b1f6c3ba8b4ed6" candidate_uuid: "e97688088d8542f7a7d708d86b197530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0a4efea0cec47928cd91fe2ec6ce02e"
I20260812 02:37:07.015967 17510 raft_consensus.cc:3037] T 4793c0463cd4487596b1f6c3ba8b4ed6 P 68b44fe628d849aea0058efe05f83434 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:37:07.015969 17349 raft_consensus.cc:3037] T 4793c0463cd4487596b1f6c3ba8b4ed6 P a0a4efea0cec47928cd91fe2ec6ce02e [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:37:07.019048 17349 raft_consensus.cc:2454] T 4793c0463cd4487596b1f6c3ba8b4ed6 P a0a4efea0cec47928cd91fe2ec6ce02e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e97688088d8542f7a7d708d86b197530 in term 1.
I20260812 02:37:07.019088 17510 raft_consensus.cc:2454] T 4793c0463cd4487596b1f6c3ba8b4ed6 P 68b44fe628d849aea0058efe05f83434 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e97688088d8542f7a7d708d86b197530 in term 1.
I20260812 02:37:07.019507 17098 leader_election.cc:302] T 4793c0463cd4487596b1f6c3ba8b4ed6 P e97688088d8542f7a7d708d86b197530 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a0a4efea0cec47928cd91fe2ec6ce02e, e97688088d8542f7a7d708d86b197530; no voters:
I20260812 02:37:07.019786 17664 raft_consensus.cc:2781] T 4793c0463cd4487596b1f6c3ba8b4ed6 P e97688088d8542f7a7d708d86b197530 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:37:07.020220 17664 raft_consensus.cc:686] T 4793c0463cd4487596b1f6c3ba8b4ed6 P e97688088d8542f7a7d708d86b197530 [term 1 LEADER]: Becoming Leader. State: Replica: e97688088d8542f7a7d708d86b197530, State: Running, Role: LEADER
I20260812 02:37:07.020490 17664 consensus_queue.cc:237] T 4793c0463cd4487596b1f6c3ba8b4ed6 P e97688088d8542f7a7d708d86b197530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0a4efea0cec47928cd91fe2ec6ce02e" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34695 } } peers { permanent_uuid: "e97688088d8542f7a7d708d86b197530" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 42457 } } peers { permanent_uuid: "68b44fe628d849aea0058efe05f83434" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33969 } }
I20260812 02:37:07.023289 16998 catalog_manager.cc:5495] T 4793c0463cd4487596b1f6c3ba8b4ed6 P e97688088d8542f7a7d708d86b197530 reported cstate change: term changed from 0 to 1, leader changed from <none> to e97688088d8542f7a7d708d86b197530 (127.7.132.129). New cstate: current_term: 1 leader_uuid: "e97688088d8542f7a7d708d86b197530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0a4efea0cec47928cd91fe2ec6ce02e" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34695 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e97688088d8542f7a7d708d86b197530" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 42457 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "68b44fe628d849aea0058efe05f83434" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33969 } health_report { overall_health: UNKNOWN } } }
I20260812 02:37:07.090979 17708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:07.085098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59320 (local address 127.7.132.129:42457)
0812 02:37:07.085775 (+ 677us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:07.085780 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:07.085794 (+ 14us) server_negotiation.cc:407] Connection header received
0812 02:37:07.085847 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:07.085850 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:07.085912 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:07.086060 (+ 148us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 02:37:07.086526 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:07.087380 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:07.088982 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:07.089414 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:07.089557 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:07.090109 (+ 552us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 02:37:07.090229 (+ 120us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 02:37:07.090337 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0812 02:37:07.090496 (+ 159us) server_negotiation.cc:299] Negotiation successful
0812 02:37:07.090579 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":540,"thread_start_us":387,"threads_started":1}
I20260812 02:37:07.100091 17510 raft_consensus.cc:1261] T 4793c0463cd4487596b1f6c3ba8b4ed6 P 68b44fe628d849aea0058efe05f83434 [term 1 FOLLOWER]: Refusing update from remote peer e97688088d8542f7a7d708d86b197530: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:37:07.100193 17349 raft_consensus.cc:1261] T 4793c0463cd4487596b1f6c3ba8b4ed6 P a0a4efea0cec47928cd91fe2ec6ce02e [term 1 FOLLOWER]: Refusing update from remote peer e97688088d8542f7a7d708d86b197530: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:37:07.100639 17702 consensus_queue.cc:1035] T 4793c0463cd4487596b1f6c3ba8b4ed6 P e97688088d8542f7a7d708d86b197530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68b44fe628d849aea0058efe05f83434" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:37:07.100996 17664 consensus_queue.cc:1035] T 4793c0463cd4487596b1f6c3ba8b4ed6 P e97688088d8542f7a7d708d86b197530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0a4efea0cec47928cd91fe2ec6ce02e" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:37:07.148113 16998 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36156:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 02:37:07.148420 16998 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36156:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 02:37:07.151486 16998 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1c4324eb32464053b83080be530c21b3: Sending DeleteTablet for 3 replicas of tablet 4793c0463cd4487596b1f6c3ba8b4ed6
I20260812 02:37:07.152456 17155 tablet_service.cc:1514] Processing DeleteTablet for tablet 4793c0463cd4487596b1f6c3ba8b4ed6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:37:07 UTC) from {username='oryx', principal='oryx/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:36814
I20260812 02:37:07.152436 17319 tablet_service.cc:1514] Processing DeleteTablet for tablet 4793c0463cd4487596b1f6c3ba8b4ed6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:37:07 UTC) from {username='oryx', principal='oryx/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:44494
I20260812 02:37:07.152618 17483 tablet_service.cc:1514] Processing DeleteTablet for tablet 4793c0463cd4487596b1f6c3ba8b4ed6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:37:07 UTC) from {username='oryx', principal='oryx/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:59376
I20260812 02:37:07.152797 17671 tablet_replica.cc:330] T 4793c0463cd4487596b1f6c3ba8b4ed6 P e97688088d8542f7a7d708d86b197530: stopping tablet replica
I20260812 02:37:07.152792 17673 tablet_replica.cc:330] T 4793c0463cd4487596b1f6c3ba8b4ed6 P a0a4efea0cec47928cd91fe2ec6ce02e: stopping tablet replica
I20260812 02:37:07.152958 17672 tablet_replica.cc:330] T 4793c0463cd4487596b1f6c3ba8b4ed6 P 68b44fe628d849aea0058efe05f83434: stopping tablet replica
I20260812 02:37:07.152994 17673 raft_consensus.cc:2229] T 4793c0463cd4487596b1f6c3ba8b4ed6 P a0a4efea0cec47928cd91fe2ec6ce02e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 02:37:07.153008 17671 raft_consensus.cc:2229] T 4793c0463cd4487596b1f6c3ba8b4ed6 P e97688088d8542f7a7d708d86b197530 [term 1 LEADER]: Raft consensus shutting down.
I20260812 02:37:07.153172 17672 raft_consensus.cc:2229] T 4793c0463cd4487596b1f6c3ba8b4ed6 P 68b44fe628d849aea0058efe05f83434 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 02:37:07.153225 17673 raft_consensus.cc:2258] T 4793c0463cd4487596b1f6c3ba8b4ed6 P a0a4efea0cec47928cd91fe2ec6ce02e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:37:07.153385 17671 raft_consensus.cc:2258] T 4793c0463cd4487596b1f6c3ba8b4ed6 P e97688088d8542f7a7d708d86b197530 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:37:07.153398 17672 raft_consensus.cc:2258] T 4793c0463cd4487596b1f6c3ba8b4ed6 P 68b44fe628d849aea0058efe05f83434 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:37:07.154290 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 17080
I20260812 02:37:07.154659 17673 ts_tablet_manager.cc:1905] T 4793c0463cd4487596b1f6c3ba8b4ed6 P a0a4efea0cec47928cd91fe2ec6ce02e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:37:07.154714 17671 ts_tablet_manager.cc:1905] T 4793c0463cd4487596b1f6c3ba8b4ed6 P e97688088d8542f7a7d708d86b197530: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:37:07.160023 17671 ts_tablet_manager.cc:1918] T 4793c0463cd4487596b1f6c3ba8b4ed6 P e97688088d8542f7a7d708d86b197530: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 02:37:07.160166 17671 log.cc:1196] T 4793c0463cd4487596b1f6c3ba8b4ed6 P e97688088d8542f7a7d708d86b197530: Deleting WAL directory at /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502156971023-7698-0/minicluster-data/ts-0/wal/wals/4793c0463cd4487596b1f6c3ba8b4ed6
I20260812 02:37:07.160692 17671 ts_tablet_manager.cc:1939] T 4793c0463cd4487596b1f6c3ba8b4ed6 P e97688088d8542f7a7d708d86b197530: Deleting consensus metadata
I20260812 02:37:07.161711 16975 catalog_manager.cc:4841] TS e97688088d8542f7a7d708d86b197530 (127.7.132.129:42457): tablet 4793c0463cd4487596b1f6c3ba8b4ed6 (table test-table [id=2a7f607ffe3647e797b210078913a6cb]) successfully deleted
I20260812 02:37:07.162987 17672 ts_tablet_manager.cc:1905] T 4793c0463cd4487596b1f6c3ba8b4ed6 P 68b44fe628d849aea0058efe05f83434: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:37:07.163079 17673 ts_tablet_manager.cc:1918] T 4793c0463cd4487596b1f6c3ba8b4ed6 P a0a4efea0cec47928cd91fe2ec6ce02e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 02:37:07.163182 17673 log.cc:1196] T 4793c0463cd4487596b1f6c3ba8b4ed6 P a0a4efea0cec47928cd91fe2ec6ce02e: Deleting WAL directory at /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502156971023-7698-0/minicluster-data/ts-1/wal/wals/4793c0463cd4487596b1f6c3ba8b4ed6
I20260812 02:37:07.163544 17673 ts_tablet_manager.cc:1939] T 4793c0463cd4487596b1f6c3ba8b4ed6 P a0a4efea0cec47928cd91fe2ec6ce02e: Deleting consensus metadata
I20260812 02:37:07.164521 16975 catalog_manager.cc:4841] TS a0a4efea0cec47928cd91fe2ec6ce02e (127.7.132.130:34695): tablet 4793c0463cd4487596b1f6c3ba8b4ed6 (table test-table [id=2a7f607ffe3647e797b210078913a6cb]) successfully deleted
I20260812 02:37:07.169407 17672 ts_tablet_manager.cc:1918] T 4793c0463cd4487596b1f6c3ba8b4ed6 P 68b44fe628d849aea0058efe05f83434: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 02:37:07.169574 17672 log.cc:1196] T 4793c0463cd4487596b1f6c3ba8b4ed6 P 68b44fe628d849aea0058efe05f83434: Deleting WAL directory at /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786502156971023-7698-0/minicluster-data/ts-2/wal/wals/4793c0463cd4487596b1f6c3ba8b4ed6
I20260812 02:37:07.170156 17672 ts_tablet_manager.cc:1939] T 4793c0463cd4487596b1f6c3ba8b4ed6 P 68b44fe628d849aea0058efe05f83434: Deleting consensus metadata
I20260812 02:37:07.171392 16980 catalog_manager.cc:4841] TS 68b44fe628d849aea0058efe05f83434 (127.7.132.131:33969): tablet 4793c0463cd4487596b1f6c3ba8b4ed6 (table test-table [id=2a7f607ffe3647e797b210078913a6cb]) successfully deleted
Aug 12 02:37:07 dist-test-slave-mn4r krb5kdc[16940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502226, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.132.129@KRBTEST.COM
I20260812 02:37:07.178526 17708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:07.155234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59324 (local address 127.7.132.129:42457)
0812 02:37:07.155406 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:07.155413 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:07.161941 (+ 6528us) server_negotiation.cc:407] Connection header received
0812 02:37:07.162250 (+ 309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:07.162255 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:07.162341 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:07.162414 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:37:07.164300 (+ 1886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:07.165269 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:07.167141 (+ 1872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:07.167570 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:07.167671 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:07.172733 (+ 5062us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:37:07.172749 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:37:07.172766 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:37:07.172814 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:37:07.175882 (+ 3068us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:37:07.176548 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:37:07.176555 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:37:07.176558 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:37:07.176698 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:37:07.177080 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:37:07.177084 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:37:07.177086 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:37:07.177451 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:37:07.177698 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0812 02:37:07.177925 (+ 227us) server_negotiation.cc:299] Negotiation successful
0812 02:37:07.178242 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 02:37:07.179216 17232 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:37:07.306499 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 17080
I20260812 02:37:07.334810 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 17243
Aug 12 02:37:07 dist-test-slave-mn4r krb5kdc[16940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502226, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.132.130@KRBTEST.COM
I20260812 02:37:07.352008 17719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:07.335612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59312 (local address 127.7.132.130:34695)
0812 02:37:07.336139 (+ 527us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:07.336164 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:07.336188 (+ 24us) server_negotiation.cc:407] Connection header received
0812 02:37:07.336263 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:07.336272 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:07.336342 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:07.336445 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:37:07.337660 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:07.338506 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:07.339754 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:07.340172 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:07.340264 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:07.344949 (+ 4685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:37:07.344967 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:37:07.344987 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:37:07.345052 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:37:07.348578 (+ 3526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:37:07.349481 (+ 903us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:37:07.349491 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:37:07.349494 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:37:07.349696 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:37:07.350215 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:37:07.350221 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:37:07.350225 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:37:07.350756 (+ 531us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:37:07.351086 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0812 02:37:07.351394 (+ 308us) server_negotiation.cc:299] Negotiation successful
0812 02:37:07.351637 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":424,"thread_start_us":323,"threads_started":1}
I20260812 02:37:07.352711 17397 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:37:07.477681 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 17243
I20260812 02:37:07.507113 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 17407
Aug 12 02:37:07 dist-test-slave-mn4r krb5kdc[16940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502226, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.132.131@KRBTEST.COM
I20260812 02:37:07.523185 17721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:07.507968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55860 (local address 127.7.132.131:33969)
0812 02:37:07.508546 (+ 578us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:07.508550 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:07.508564 (+ 14us) server_negotiation.cc:407] Connection header received
0812 02:37:07.508620 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:07.508623 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:07.508681 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:07.508801 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:37:07.510178 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:07.510905 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:07.512130 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:07.512627 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:07.512758 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:07.517263 (+ 4505us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:37:07.517279 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:37:07.517296 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:37:07.517328 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:37:07.520269 (+ 2941us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:37:07.520928 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:37:07.520937 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:37:07.520940 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:37:07.521093 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:37:07.521455 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:37:07.521460 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:37:07.521468 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:37:07.521816 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:37:07.522188 (+ 372us) server_negotiation.cc:1035] Waiting for connection context
0812 02:37:07.522569 (+ 381us) server_negotiation.cc:299] Negotiation successful
0812 02:37:07.522800 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":463,"thread_start_us":299,"threads_started":1}
I20260812 02:37:07.523877 17557 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:37:07.647445 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 17407
I20260812 02:37:07.676247 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 16956
I20260812 02:37:07.677446 17044 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:37:07.793882 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 16956
2026-08-12T02:37:07Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (7183 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786502156971023-7698-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 02:37:07 dist-test-slave-mn4r krb5kdc[17727](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 02:37:07 dist-test-slave-mn4r krb5kdc[17727](info): set up 2 sockets
Aug 12 02:37:07 dist-test-slave-mn4r krb5kdc[17727](info): commencing operation
krb5kdc: starting...
W20260812 02:37:09.021287 7698 mini_kdc.cc:121] Time spent starting KDC: real 1.177s user 0.001s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786502156971023-7698-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786502156971023-7698-0/krb5kdc/test-user.keytab.
Aug 12 02:37:09 dist-test-slave-mn4r krb5kdc[17727](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502229, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T02:37:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T02:37:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.132.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
I20260812 02:37:09.289673 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786502156971023-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786502156971023-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786502156971023-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:44815
--webserver_interface=127.7.132.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:41309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.132.190:44815
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:37:09.654356 17744 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:37:09.654712 17744 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:37:09.654884 17744 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:37:09.662058 17744 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:37:09.662238 17744 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:37:09.662343 17744 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:37:09.662441 17744 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:37:09.662511 17744 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:37:09.672534 17744 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:41309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786502156971023-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786502156971023-7698-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.132.190:44815
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:44815
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.132.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786502156971023-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:37:09.673971 17744 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:37:09.675379 17744 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:37:09.687564 17749 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:37:09.687525 17752 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:37:09.687512 17750 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:37:09.688527 17744 server_base.cc:1034] running on GCE node
I20260812 02:37:09.689361 17744 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:37:09.690886 17744 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:37:09.692121 17744 hybrid_clock.cc:648] HybridClock initialized: now 1786502229692069 us; error 67 us; skew 500 ppm
I20260812 02:37:09.692476 17744 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-12T02:37:09Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1973 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 02:37:09 dist-test-slave-mn4r krb5kdc[17758](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 02:37:09 dist-test-slave-mn4r krb5kdc[17758](info): set up 2 sockets
Aug 12 02:37:09 dist-test-slave-mn4r krb5kdc[17758](info): commencing operation
krb5kdc: starting...
W20260812 02:37:10.987576 7698 mini_kdc.cc:121] Time spent starting KDC: real 1.170s user 0.004s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/krb5kdc/test-user.keytab.
Aug 12 02:37:11 dist-test-slave-mn4r krb5kdc[17758](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502231, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T02:37:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T02:37:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.132.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
I20260812 02:37:11.262398 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:37037
--webserver_interface=127.7.132.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:35499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.132.190:37037
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:37:11.613875 17771 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:37:11.614203 17771 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:37:11.614395 17771 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:37:11.623142 17771 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:37:11.623255 17771 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:37:11.623358 17771 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:37:11.623464 17771 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:37:11.623540 17771 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:37:11.633502 17771 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:35499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.132.190:37037
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:37037
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.132.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:37:11.634939 17771 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:37:11.636369 17771 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:37:11.648623 17776 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:37:11.648661 17781 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:37:11.648711 17778 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:37:11.649590 17771 server_base.cc:1034] running on GCE node
I20260812 02:37:11.650441 17771 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:37:11.651924 17771 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:37:11.653182 17771 hybrid_clock.cc:648] HybridClock initialized: now 1786502231653139 us; error 55 us; skew 500 ppm
I20260812 02:37:11.653551 17771 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:37:11 dist-test-slave-mn4r krb5kdc[17758](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502231, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:37:11.656782 17771 init.cc:377] Logged in from keytab as kudu/127.7.132.190@KRBTEST.COM (short username kudu)
I20260812 02:37:11.658349 17771 webserver.cc:460] Webserver started at http://127.7.132.190:36257/ using document root <none> and password file <none>
I20260812 02:37:11.659018 17771 fs_manager.cc:359] Metadata directory not provided
I20260812 02:37:11.659126 17771 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:37:11.659391 17771 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:37:11.662528 17771 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/master-0/data/instance:
uuid: "4192d9eb1ca44723996b9cee0b1ec577"
format_stamp: "Formatted at 2026-08-12 02:37:11 on dist-test-slave-mn4r"
server_key: "6da6408bd5bde6de8f1e871bd6a54b9d"
server_key_iv: "cd71c0360550febcb87ebef5794b134d"
server_key_version: "encryptionkey@0"
I20260812 02:37:11.663354 17771 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/master-0/wal/instance:
uuid: "4192d9eb1ca44723996b9cee0b1ec577"
format_stamp: "Formatted at 2026-08-12 02:37:11 on dist-test-slave-mn4r"
server_key: "6da6408bd5bde6de8f1e871bd6a54b9d"
server_key_iv: "cd71c0360550febcb87ebef5794b134d"
server_key_version: "encryptionkey@0"
I20260812 02:37:11.668550 17771 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 02:37:11.672164 17789 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:37:11.673599 17771 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 02:37:11.673781 17771 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/master-0/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/master-0/wal
uuid: "4192d9eb1ca44723996b9cee0b1ec577"
format_stamp: "Formatted at 2026-08-12 02:37:11 on dist-test-slave-mn4r"
server_key: "6da6408bd5bde6de8f1e871bd6a54b9d"
server_key_iv: "cd71c0360550febcb87ebef5794b134d"
server_key_version: "encryptionkey@0"
I20260812 02:37:11.673969 17771 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:37:11.687381 17771 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:37:11.691605 17771 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:37:11.691926 17771 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:37:11.711943 17771 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.190:37037
I20260812 02:37:11.711963 17851 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.190:37037 every 8 connection(s)
I20260812 02:37:11.713502 17771 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
I20260812 02:37:11.717641 17852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:37:11.721082 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 17771
I20260812 02:37:11.721306 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/master-0/wal/instance
I20260812 02:37:11.722216 7698 external_mini_cluster.cc:1428] Setting key 478c6aa1ff97ccf4a534ad31fc8f61b7
I20260812 02:37:11.731403 17852 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4192d9eb1ca44723996b9cee0b1ec577: Bootstrap starting.
I20260812 02:37:11.735371 17852 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4192d9eb1ca44723996b9cee0b1ec577: Neither blocks nor log segments found. Creating new log.
I20260812 02:37:11.736857 17852 log.cc:824] T 00000000000000000000000000000000 P 4192d9eb1ca44723996b9cee0b1ec577: Log is configured to *not* fsync() on all Append() calls
Aug 12 02:37:11 dist-test-slave-mn4r krb5kdc[17758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502231, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 02:37:11.739980 17852 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4192d9eb1ca44723996b9cee0b1ec577: No bootstrap required, opened a new log
I20260812 02:37:11.746867 17858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:11.724416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41322 (local address 127.7.132.190:37037)
0812 02:37:11.725258 (+ 842us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:11.725270 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:11.725314 (+ 44us) server_negotiation.cc:407] Connection header received
0812 02:37:11.727071 (+ 1757us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:11.727095 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:11.727532 (+ 437us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:11.727893 (+ 361us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:37:11.729823 (+ 1930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:11.731213 (+ 1390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:11.732969 (+ 1756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:11.733673 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:11.733813 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:11.738965 (+ 5152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:37:11.738973 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:37:11.738989 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:37:11.739026 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:37:11.742362 (+ 3336us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:37:11.743143 (+ 781us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:37:11.743151 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:37:11.743157 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:37:11.743334 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:37:11.743788 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:37:11.743795 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:37:11.743802 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:37:11.744395 (+ 593us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:37:11.744745 (+ 350us) server_negotiation.cc:1035] Waiting for connection context
0812 02:37:11.745221 (+ 476us) server_negotiation.cc:299] Negotiation successful
0812 02:37:11.745702 (+ 481us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":613,"thread_start_us":392,"threads_started":1}
I20260812 02:37:11.748410 17852 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4192d9eb1ca44723996b9cee0b1ec577 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4192d9eb1ca44723996b9cee0b1ec577" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 37037 } }
I20260812 02:37:11.748818 17852 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4192d9eb1ca44723996b9cee0b1ec577 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:37:11.748935 17852 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4192d9eb1ca44723996b9cee0b1ec577 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4192d9eb1ca44723996b9cee0b1ec577, State: Initialized, Role: FOLLOWER
I20260812 02:37:11.749554 17852 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4192d9eb1ca44723996b9cee0b1ec577 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4192d9eb1ca44723996b9cee0b1ec577" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 37037 } }
I20260812 02:37:11.749815 17852 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4192d9eb1ca44723996b9cee0b1ec577 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 02:37:11.749965 17852 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4192d9eb1ca44723996b9cee0b1ec577 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 02:37:11.750183 17852 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4192d9eb1ca44723996b9cee0b1ec577 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:37:11.752696 17852 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4192d9eb1ca44723996b9cee0b1ec577 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4192d9eb1ca44723996b9cee0b1ec577" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 37037 } }
I20260812 02:37:11.753149 17852 leader_election.cc:302] T 00000000000000000000000000000000 P 4192d9eb1ca44723996b9cee0b1ec577 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4192d9eb1ca44723996b9cee0b1ec577; no voters:
I20260812 02:37:11.753655 17852 leader_election.cc:288] T 00000000000000000000000000000000 P 4192d9eb1ca44723996b9cee0b1ec577 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 02:37:11.753803 17861 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4192d9eb1ca44723996b9cee0b1ec577 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:37:11.754537 17861 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4192d9eb1ca44723996b9cee0b1ec577 [term 1 LEADER]: Becoming Leader. State: Replica: 4192d9eb1ca44723996b9cee0b1ec577, State: Running, Role: LEADER
I20260812 02:37:11.755120 17861 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4192d9eb1ca44723996b9cee0b1ec577 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4192d9eb1ca44723996b9cee0b1ec577" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 37037 } }
I20260812 02:37:11.755710 17852 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4192d9eb1ca44723996b9cee0b1ec577 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:37:11.758206 17863 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4192d9eb1ca44723996b9cee0b1ec577 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4192d9eb1ca44723996b9cee0b1ec577. Latest consensus state: current_term: 1 leader_uuid: "4192d9eb1ca44723996b9cee0b1ec577" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4192d9eb1ca44723996b9cee0b1ec577" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 37037 } } }
I20260812 02:37:11.758602 17863 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4192d9eb1ca44723996b9cee0b1ec577 [sys.catalog]: This master's current role is: LEADER
I20260812 02:37:11.758522 17862 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4192d9eb1ca44723996b9cee0b1ec577 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4192d9eb1ca44723996b9cee0b1ec577" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4192d9eb1ca44723996b9cee0b1ec577" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 37037 } } }
I20260812 02:37:11.758814 17862 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4192d9eb1ca44723996b9cee0b1ec577 [sys.catalog]: This master's current role is: LEADER
I20260812 02:37:11.760936 17869 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:37:11.766247 17869 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 02:37:11.776149 17869 catalog_manager.cc:1347] Generated new cluster ID: dd86a88a7e65484cb3da1aced5b5e7c7
I20260812 02:37:11.776274 17869 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 02:37:11.785722 17869 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 02:37:11.787774 17869 catalog_manager.cc:1504] Loading token signing keys...
I20260812 02:37:11.796954 17869 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4192d9eb1ca44723996b9cee0b1ec577: Generated new TSK 0
I20260812 02:37:11.798064 17869 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.132.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.129.keytab.
Entry for principal kudu/127.7.132.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.129.keytab.
I20260812 02:37:11.865782 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.129:0
--local_ip_for_outbound_sockets=127.7.132.129
--webserver_interface=127.7.132.129
--webserver_port=0
--tserver_master_addrs=127.7.132.190:37037
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.129.keytab
--principal=kudu/127.7.132.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:35499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:37:12.217746 17889 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:37:12.218048 17889 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:37:12.218278 17889 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:37:12.225806 17889 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:37:12.225916 17889 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:37:12.226177 17889 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.129
I20260812 02:37:12.236310 17889 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:35499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.129.keytab
--principal=kudu/127.7.132.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.132.129
--webserver_port=0
--tserver_master_addrs=127.7.132.190:37037
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.129
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:37:12.237742 17889 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:37:12.239254 17889 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:37:12.253064 17897 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:37:12.253140 17900 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:37:12.253150 17898 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:37:12.254316 17889 server_base.cc:1034] running on GCE node
I20260812 02:37:12.254995 17889 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:37:12.256057 17889 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:37:12.257298 17889 hybrid_clock.cc:648] HybridClock initialized: now 1786502232257250 us; error 56 us; skew 500 ppm
I20260812 02:37:12.257645 17889 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:37:12 dist-test-slave-mn4r krb5kdc[17758](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502232, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:37:12.261039 17889 init.cc:377] Logged in from keytab as kudu/127.7.132.129@KRBTEST.COM (short username kudu)
I20260812 02:37:12.262663 17889 webserver.cc:460] Webserver started at http://127.7.132.129:34249/ using document root <none> and password file <none>
I20260812 02:37:12.263352 17889 fs_manager.cc:359] Metadata directory not provided
I20260812 02:37:12.263454 17889 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:37:12.263731 17889 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:37:12.266750 17889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/ts-0/data/instance:
uuid: "77cfaeb2c06b407db5b8e28ad5ad901d"
format_stamp: "Formatted at 2026-08-12 02:37:12 on dist-test-slave-mn4r"
server_key: "af8b67e8d29dad007cd68cb06a5ff150"
server_key_iv: "f68529c6fa6d14e03f046b2fef6e33d6"
server_key_version: "encryptionkey@0"
I20260812 02:37:12.267522 17889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/ts-0/wal/instance:
uuid: "77cfaeb2c06b407db5b8e28ad5ad901d"
format_stamp: "Formatted at 2026-08-12 02:37:12 on dist-test-slave-mn4r"
server_key: "af8b67e8d29dad007cd68cb06a5ff150"
server_key_iv: "f68529c6fa6d14e03f046b2fef6e33d6"
server_key_version: "encryptionkey@0"
I20260812 02:37:12.272864 17889 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260812 02:37:12.276598 17908 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:37:12.277918 17889 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260812 02:37:12.278162 17889 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/ts-0/wal
uuid: "77cfaeb2c06b407db5b8e28ad5ad901d"
format_stamp: "Formatted at 2026-08-12 02:37:12 on dist-test-slave-mn4r"
server_key: "af8b67e8d29dad007cd68cb06a5ff150"
server_key_iv: "f68529c6fa6d14e03f046b2fef6e33d6"
server_key_version: "encryptionkey@0"
I20260812 02:37:12.278374 17889 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:37:12.305063 17889 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:37:12.309739 17889 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:37:12.310055 17889 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:37:12.311118 17889 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:37:12.312505 17889 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:37:12.312608 17889 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:37:12.312770 17889 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:37:12.312844 17889 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:37:12.347498 17889 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.129:39069
I20260812 02:37:12.347546 18035 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.129:39069 every 8 connection(s)
I20260812 02:37:12.348843 17889 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/ts-0/data/info.pb
I20260812 02:37:12.353932 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 17889
I20260812 02:37:12.354171 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/ts-0/wal/instance
I20260812 02:37:12.354826 7698 external_mini_cluster.cc:1428] Setting key 85a14dc2f8b7872a56fca69a4075db7a
Aug 12 02:37:12 dist-test-slave-mn4r krb5kdc[17758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502232, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.129@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 02:37:12.371845 18040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:12.351580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:44707 (local address 127.7.132.190:37037)
0812 02:37:12.352054 (+ 474us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:12.352059 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:12.353047 (+ 988us) server_negotiation.cc:407] Connection header received
0812 02:37:12.355118 (+ 2071us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:12.355124 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:12.355221 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:12.355346 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:37:12.357267 (+ 1921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:12.358009 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:12.359667 (+ 1658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:12.360167 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:12.360321 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:12.364810 (+ 4489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:37:12.364819 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:37:12.364827 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:37:12.364875 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:37:12.368276 (+ 3401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:37:12.369347 (+ 1071us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:37:12.369351 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:37:12.369353 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:37:12.369476 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:37:12.369977 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:37:12.369981 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:37:12.369983 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:37:12.370331 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:37:12.370449 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0812 02:37:12.371391 (+ 942us) server_negotiation.cc:299] Negotiation successful
0812 02:37:12.371569 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":364,"thread_start_us":233,"threads_started":1}
I20260812 02:37:12.373543 18041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:12.352147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:37037 (local address 127.7.132.129:44707)
0812 02:37:12.352848 (+ 701us) negotiation.cc:107] Waiting for socket to connect
0812 02:37:12.352890 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 02:37:12.354778 (+ 1888us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:37:12.355684 (+ 906us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:37:12.355699 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:37:12.356309 (+ 610us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:37:12.357075 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:12.357113 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:12.358210 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:12.358216 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:12.359499 (+ 1283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:12.359513 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:12.360452 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:12.360461 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:12.360711 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:12.361578 (+ 867us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:37:12.361615 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:37:12.364540 (+ 2925us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:37:12.368441 (+ 3901us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:37:12.368454 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:37:12.368476 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:37:12.369219 (+ 743us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:37:12.369587 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:37:12.369595 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:37:12.369602 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:37:12.369863 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:37:12.370463 (+ 600us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:37:12.370474 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:37:12.370958 (+ 484us) client_negotiation.cc:769] Sending connection context
0812 02:37:12.371300 (+ 342us) client_negotiation.cc:240] Negotiation successful
0812 02:37:12.371960 (+ 660us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":267,"threads_started":1}
I20260812 02:37:12.375418 18036 heartbeater.cc:347] Connected to a master server at 127.7.132.190:37037
I20260812 02:37:12.375770 18036 heartbeater.cc:464] Registering TS with master...
I20260812 02:37:12.376554 18036 heartbeater.cc:511] Master 127.7.132.190:37037 requested a full tablet report, sending...
I20260812 02:37:12.379009 17809 ts_manager.cc:194] Registered new tserver with Master: 77cfaeb2c06b407db5b8e28ad5ad901d (127.7.132.129:39069)
I20260812 02:37:12.380645 17809 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.129@KRBTEST.COM'} at 127.7.132.129:44707
WARNING: no policy specified for kudu/127.7.132.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.130.keytab.
Entry for principal kudu/127.7.132.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.130.keytab.
I20260812 02:37:12.417418 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.130:0
--local_ip_for_outbound_sockets=127.7.132.130
--webserver_interface=127.7.132.130
--webserver_port=0
--tserver_master_addrs=127.7.132.190:37037
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.130.keytab
--principal=kudu/127.7.132.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:35499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:37:12.769376 18044 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:37:12.769727 18044 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:37:12.769896 18044 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:37:12.778242 18044 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:37:12.778388 18044 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:37:12.778609 18044 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.130
I20260812 02:37:12.788851 18044 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:35499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.130.keytab
--principal=kudu/127.7.132.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.132.130
--webserver_port=0
--tserver_master_addrs=127.7.132.190:37037
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.130
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:37:12.790341 18044 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:37:12.791764 18044 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:37:12.806357 18050 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:37:12.806346 18053 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:37:12.806389 18051 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:37:12.807302 18044 server_base.cc:1034] running on GCE node
I20260812 02:37:12.807886 18044 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:37:12.808830 18044 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:37:12.810115 18044 hybrid_clock.cc:648] HybridClock initialized: now 1786502232810039 us; error 54 us; skew 500 ppm
I20260812 02:37:12.810468 18044 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:37:12 dist-test-slave-mn4r krb5kdc[17758](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502232, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:37:12.813637 18044 init.cc:377] Logged in from keytab as kudu/127.7.132.130@KRBTEST.COM (short username kudu)
I20260812 02:37:12.815100 18044 webserver.cc:460] Webserver started at http://127.7.132.130:44897/ using document root <none> and password file <none>
I20260812 02:37:12.815762 18044 fs_manager.cc:359] Metadata directory not provided
I20260812 02:37:12.815863 18044 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:37:12.816165 18044 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:37:12.819165 18044 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/ts-1/data/instance:
uuid: "d8915b74441f419db7f678b24f1b6ccb"
format_stamp: "Formatted at 2026-08-12 02:37:12 on dist-test-slave-mn4r"
server_key: "e25b3f2d18c2d055d0503b5d7b0afed4"
server_key_iv: "3c29eda64a705bb532f0987d34b054a5"
server_key_version: "encryptionkey@0"
I20260812 02:37:12.819921 18044 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/ts-1/wal/instance:
uuid: "d8915b74441f419db7f678b24f1b6ccb"
format_stamp: "Formatted at 2026-08-12 02:37:12 on dist-test-slave-mn4r"
server_key: "e25b3f2d18c2d055d0503b5d7b0afed4"
server_key_iv: "3c29eda64a705bb532f0987d34b054a5"
server_key_version: "encryptionkey@0"
I20260812 02:37:12.825034 18044 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260812 02:37:12.828456 18063 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:37:12.829780 18044 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260812 02:37:12.829916 18044 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/ts-1/wal
uuid: "d8915b74441f419db7f678b24f1b6ccb"
format_stamp: "Formatted at 2026-08-12 02:37:12 on dist-test-slave-mn4r"
server_key: "e25b3f2d18c2d055d0503b5d7b0afed4"
server_key_iv: "3c29eda64a705bb532f0987d34b054a5"
server_key_version: "encryptionkey@0"
I20260812 02:37:12.830134 18044 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:37:12.846925 18044 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:37:12.851325 18044 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:37:12.851572 18044 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:37:12.852526 18044 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:37:12.853901 18044 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:37:12.853974 18044 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:37:12.854063 18044 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:37:12.854185 18044 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260812 02:37:12.889837 18044 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.130:42401
I20260812 02:37:12.889887 18198 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.130:42401 every 8 connection(s)
I20260812 02:37:12.891219 18044 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/ts-1/data/info.pb
I20260812 02:37:12.895510 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 18044
I20260812 02:37:12.895701 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/ts-1/wal/instance
I20260812 02:37:12.896304 7698 external_mini_cluster.cc:1428] Setting key c871150732e8fa7ffa7a11775120d4fe
Aug 12 02:37:12 dist-test-slave-mn4r krb5kdc[17758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502232, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.130@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 02:37:12.914865 18202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:12.893890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:37977 (local address 127.7.132.190:37037)
0812 02:37:12.894415 (+ 525us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:12.894420 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:12.895513 (+ 1093us) server_negotiation.cc:407] Connection header received
0812 02:37:12.897922 (+ 2409us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:12.897927 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:12.897993 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:12.898157 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:37:12.900247 (+ 2090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:12.900976 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:12.903564 (+ 2588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:12.903985 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:12.904089 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:12.908489 (+ 4400us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:37:12.908494 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:37:12.908498 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:37:12.908532 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:37:12.911708 (+ 3176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:37:12.912443 (+ 735us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:37:12.912447 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:37:12.912449 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:37:12.912575 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:37:12.913029 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:37:12.913039 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:37:12.913041 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:37:12.913328 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:37:12.913488 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0812 02:37:12.914246 (+ 758us) server_negotiation.cc:299] Negotiation successful
0812 02:37:12.914474 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":411,"thread_start_us":297,"threads_started":1}
I20260812 02:37:12.915971 18203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:12.894445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:37037 (local address 127.7.132.130:37977)
0812 02:37:12.895295 (+ 850us) negotiation.cc:107] Waiting for socket to connect
0812 02:37:12.895359 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0812 02:37:12.897595 (+ 2236us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:37:12.898433 (+ 838us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:37:12.898453 (+ 20us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:37:12.899173 (+ 720us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:37:12.900040 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:12.900072 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:12.901151 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:12.901159 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:12.903407 (+ 2248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:12.903428 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:12.904219 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:12.904230 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:12.904576 (+ 346us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:12.905366 (+ 790us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:37:12.905401 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:37:12.908238 (+ 2837us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:37:12.911846 (+ 3608us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:37:12.911855 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:37:12.911872 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:37:12.912299 (+ 427us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:37:12.912689 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:37:12.912694 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:37:12.912698 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:37:12.912899 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:37:12.913464 (+ 565us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:37:12.913473 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:37:12.913837 (+ 364us) client_negotiation.cc:769] Sending connection context
0812 02:37:12.914157 (+ 320us) client_negotiation.cc:240] Negotiation successful
0812 02:37:12.914731 (+ 574us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":276,"threads_started":1}
I20260812 02:37:12.917588 18199 heartbeater.cc:347] Connected to a master server at 127.7.132.190:37037
I20260812 02:37:12.917908 18199 heartbeater.cc:464] Registering TS with master...
I20260812 02:37:12.918700 18199 heartbeater.cc:511] Master 127.7.132.190:37037 requested a full tablet report, sending...
I20260812 02:37:12.920279 17809 ts_manager.cc:194] Registered new tserver with Master: d8915b74441f419db7f678b24f1b6ccb (127.7.132.130:42401)
I20260812 02:37:12.921656 17809 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.130@KRBTEST.COM'} at 127.7.132.130:37977
WARNING: no policy specified for kudu/127.7.132.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.131.keytab.
Entry for principal kudu/127.7.132.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.131.keytab.
I20260812 02:37:12.961102 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.131:0
--local_ip_for_outbound_sockets=127.7.132.131
--webserver_interface=127.7.132.131
--webserver_port=0
--tserver_master_addrs=127.7.132.190:37037
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.131.keytab
--principal=kudu/127.7.132.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:35499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:37:13.315992 18207 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:37:13.316262 18207 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:37:13.316358 18207 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:37:13.323689 18207 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:37:13.323844 18207 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:37:13.324060 18207 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.131
I20260812 02:37:13.333767 18207 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:35499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.131.keytab
--principal=kudu/127.7.132.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.132.131
--webserver_port=0
--tserver_master_addrs=127.7.132.190:37037
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.131
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:37:13.335234 18207 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:37:13.336616 18207 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:37:13.352093 18216 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:37:13.352125 18218 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:37:13.352826 18215 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:37:13.353618 18207 server_base.cc:1034] running on GCE node
I20260812 02:37:13.354296 18207 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:37:13.355228 18207 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:37:13.356472 18207 hybrid_clock.cc:648] HybridClock initialized: now 1786502233356435 us; error 50 us; skew 500 ppm
I20260812 02:37:13.356796 18207 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:37:13 dist-test-slave-mn4r krb5kdc[17758](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502233, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:37:13.360140 18207 init.cc:377] Logged in from keytab as kudu/127.7.132.131@KRBTEST.COM (short username kudu)
I20260812 02:37:13.361984 18207 webserver.cc:460] Webserver started at http://127.7.132.131:40203/ using document root <none> and password file <none>
I20260812 02:37:13.362708 18207 fs_manager.cc:359] Metadata directory not provided
I20260812 02:37:13.362855 18207 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:37:13.363188 18207 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:37:13.366171 18207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/ts-2/data/instance:
uuid: "cf17c7e326334289a829888d92d40ce7"
format_stamp: "Formatted at 2026-08-12 02:37:13 on dist-test-slave-mn4r"
server_key: "71d9735807f6be7736346a50dd7088fe"
server_key_iv: "71dd8c7d9fe2f8401986aba64f97b4f2"
server_key_version: "encryptionkey@0"
I20260812 02:37:13.367012 18207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/ts-2/wal/instance:
uuid: "cf17c7e326334289a829888d92d40ce7"
format_stamp: "Formatted at 2026-08-12 02:37:13 on dist-test-slave-mn4r"
server_key: "71d9735807f6be7736346a50dd7088fe"
server_key_iv: "71dd8c7d9fe2f8401986aba64f97b4f2"
server_key_version: "encryptionkey@0"
I20260812 02:37:13.372325 18207 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260812 02:37:13.376722 18225 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:37:13.378334 18207 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260812 02:37:13.378506 18207 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/ts-2/wal
uuid: "cf17c7e326334289a829888d92d40ce7"
format_stamp: "Formatted at 2026-08-12 02:37:13 on dist-test-slave-mn4r"
server_key: "71d9735807f6be7736346a50dd7088fe"
server_key_iv: "71dd8c7d9fe2f8401986aba64f97b4f2"
server_key_version: "encryptionkey@0"
I20260812 02:37:13.378717 18207 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:37:13.383870 18036 heartbeater.cc:503] Master 127.7.132.190:37037 was elected leader, sending a full tablet report...
I20260812 02:37:13.399948 18207 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:37:13.404489 18207 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:37:13.404723 18207 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:37:13.405676 18207 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:37:13.407043 18207 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:37:13.407119 18207 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:37:13.407200 18207 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:37:13.407245 18207 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:37:13.442695 18207 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.131:44117
I20260812 02:37:13.442773 18367 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.131:44117 every 8 connection(s)
I20260812 02:37:13.444074 18207 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/ts-2/data/info.pb
I20260812 02:37:13.449028 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 18207
I20260812 02:37:13.449208 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/ts-2/wal/instance
I20260812 02:37:13.449690 7698 external_mini_cluster.cc:1428] Setting key 5bf359722ddc945d1c1e407af75aa2d4
Aug 12 02:37:13 dist-test-slave-mn4r krb5kdc[17758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502233, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.131@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 02:37:13.465999 18371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:13.446847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:52701 (local address 127.7.132.190:37037)
0812 02:37:13.447369 (+ 522us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:13.447373 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:13.448318 (+ 945us) server_negotiation.cc:407] Connection header received
0812 02:37:13.450538 (+ 2220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:13.450544 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:13.450630 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:13.450753 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:37:13.452855 (+ 2102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:13.453751 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:13.455437 (+ 1686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:13.455937 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:13.456069 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:13.459657 (+ 3588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:37:13.459662 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:37:13.459665 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:37:13.459699 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:37:13.462481 (+ 2782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:37:13.463238 (+ 757us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:37:13.463243 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:37:13.463246 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:37:13.463372 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:37:13.464054 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:37:13.464060 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:37:13.464063 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:37:13.464441 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:37:13.464591 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0812 02:37:13.465519 (+ 928us) server_negotiation.cc:299] Negotiation successful
0812 02:37:13.465695 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":236,"threads_started":1}
I20260812 02:37:13.466974 18372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:13.447424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:37037 (local address 127.7.132.131:52701)
0812 02:37:13.448149 (+ 725us) negotiation.cc:107] Waiting for socket to connect
0812 02:37:13.448196 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 02:37:13.450186 (+ 1990us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:37:13.451106 (+ 920us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:37:13.451117 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:37:13.451798 (+ 681us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:37:13.452653 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:13.452683 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:13.453911 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:13.453916 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:13.455224 (+ 1308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:13.455280 (+ 56us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:13.456168 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:13.456175 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:13.456497 (+ 322us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:13.457141 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:37:13.457165 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:37:13.459442 (+ 2277us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:37:13.462624 (+ 3182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:37:13.462633 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:37:13.462649 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:37:13.463081 (+ 432us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:37:13.463661 (+ 580us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:37:13.463664 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:37:13.463667 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:37:13.463905 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:37:13.464601 (+ 696us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:37:13.464611 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:37:13.465020 (+ 409us) client_negotiation.cc:769] Sending connection context
0812 02:37:13.465375 (+ 355us) client_negotiation.cc:240] Negotiation successful
0812 02:37:13.465794 (+ 419us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":281,"threads_started":1}
I20260812 02:37:13.468513 18369 heartbeater.cc:347] Connected to a master server at 127.7.132.190:37037
I20260812 02:37:13.468791 18369 heartbeater.cc:464] Registering TS with master...
I20260812 02:37:13.469534 18369 heartbeater.cc:511] Master 127.7.132.190:37037 requested a full tablet report, sending...
I20260812 02:37:13.471050 17809 ts_manager.cc:194] Registered new tserver with Master: cf17c7e326334289a829888d92d40ce7 (127.7.132.131:44117)
I20260812 02:37:13.471828 17809 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.131@KRBTEST.COM'} at 127.7.132.131:52701
I20260812 02:37:13.477713 7698 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.7.132.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.132.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.132.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/krb5kdc/oryx_127.7.132.129.keytab.
Entry for principal oryx/127.7.132.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/krb5kdc/oryx_127.7.132.129.keytab.
WARNING: no policy specified for oryx/127.7.132.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.132.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.132.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/krb5kdc/oryx_127.7.132.130.keytab.
Entry for principal oryx/127.7.132.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/krb5kdc/oryx_127.7.132.130.keytab.
WARNING: no policy specified for oryx/127.7.132.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.132.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.132.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/krb5kdc/oryx_127.7.132.131.keytab.
Entry for principal oryx/127.7.132.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/krb5kdc/oryx_127.7.132.131.keytab.
I20260812 02:37:13.644306 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 17889
Aug 12 02:37:13 dist-test-slave-mn4r krb5kdc[17758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502231, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.132.129@KRBTEST.COM
I20260812 02:37:13.661948 18381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:13.645484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41208 (local address 127.7.132.129:39069)
0812 02:37:13.646199 (+ 715us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:13.646218 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:13.646251 (+ 33us) server_negotiation.cc:407] Connection header received
0812 02:37:13.646408 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:13.646416 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:13.646650 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:13.646871 (+ 221us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:37:13.648323 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:13.649352 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:13.650730 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:13.651329 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:13.651471 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:13.655772 (+ 4301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:37:13.655785 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:37:13.655800 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:37:13.655834 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:37:13.658911 (+ 3077us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:37:13.659611 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:37:13.659619 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:37:13.659622 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:37:13.659753 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:37:13.660195 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:37:13.660200 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:37:13.660203 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:37:13.660556 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:37:13.660928 (+ 372us) server_negotiation.cc:1035] Waiting for connection context
0812 02:37:13.661295 (+ 367us) server_negotiation.cc:299] Negotiation successful
0812 02:37:13.661573 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":524,"thread_start_us":335,"threads_started":1}
I20260812 02:37:13.663269 18031 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:37:13.753551 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 17889
I20260812 02:37:13.777405 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 18044
Aug 12 02:37:13 dist-test-slave-mn4r krb5kdc[17758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502231, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.132.130@KRBTEST.COM
I20260812 02:37:13.794983 18383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:13.778608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56658 (local address 127.7.132.130:42401)
0812 02:37:13.779244 (+ 636us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:13.779251 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:13.779277 (+ 26us) server_negotiation.cc:407] Connection header received
0812 02:37:13.779340 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:13.779347 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:13.779563 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:13.779750 (+ 187us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:37:13.781094 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:13.782577 (+ 1483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:13.784057 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:13.784641 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:13.784790 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:13.789325 (+ 4535us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:37:13.789338 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:37:13.789353 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:37:13.789392 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:37:13.792414 (+ 3022us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:37:13.793068 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:37:13.793075 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:37:13.793077 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:37:13.793205 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:37:13.793517 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:37:13.793520 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:37:13.793522 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:37:13.793867 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:37:13.794169 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0812 02:37:13.794443 (+ 274us) server_negotiation.cc:299] Negotiation successful
0812 02:37:13.794651 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":473,"thread_start_us":301,"threads_started":1}
I20260812 02:37:13.796336 18193 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:37:13.879845 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 18044
I20260812 02:37:13.900774 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 18207
Aug 12 02:37:13 dist-test-slave-mn4r krb5kdc[17758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502231, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.132.131@KRBTEST.COM
I20260812 02:37:13.918051 18387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:13.901927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47896 (local address 127.7.132.131:44117)
0812 02:37:13.902521 (+ 594us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:13.902529 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:13.902559 (+ 30us) server_negotiation.cc:407] Connection header received
0812 02:37:13.902620 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:13.902626 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:13.902839 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:13.902980 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:37:13.904317 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:13.905364 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:13.906796 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:13.907374 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:13.907485 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:13.911982 (+ 4497us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:37:13.911996 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:37:13.912011 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:37:13.912056 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:37:13.915202 (+ 3146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:37:13.915900 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:37:13.915939 (+ 39us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:37:13.915942 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:37:13.916084 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:37:13.916458 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:37:13.916462 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:37:13.916464 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:37:13.916900 (+ 436us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:37:13.917133 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0812 02:37:13.917466 (+ 333us) server_negotiation.cc:299] Negotiation successful
0812 02:37:13.917692 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":445,"thread_start_us":316,"threads_started":1}
I20260812 02:37:13.919565 18363 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:37:14.003294 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 18207
I20260812 02:37:14.025211 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 17771
I20260812 02:37:14.026554 17846 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:37:14.133249 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 17771
I20260812 02:37:14.159192 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:37037
--webserver_interface=127.7.132.190
--webserver_port=36257
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:35499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.132.190:37037
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:37:14.501713 18390 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:37:14.502130 18390 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:37:14.502312 18390 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:37:14.509748 18390 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:37:14.509869 18390 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:37:14.509958 18390 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:37:14.510105 18390 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:37:14.510201 18390 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:37:14.521140 18390 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:35499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.132.190:37037
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:37037
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.132.190
--webserver_port=36257
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:37:14.522611 18390 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:37:14.523983 18390 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:37:14.536451 18398 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:37:14.536386 18396 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:37:14.536406 18395 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:37:14.537310 18390 server_base.cc:1034] running on GCE node
I20260812 02:37:14.538153 18390 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:37:14.539623 18390 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:37:14.540848 18390 hybrid_clock.cc:648] HybridClock initialized: now 1786502234540800 us; error 40 us; skew 500 ppm
I20260812 02:37:14.541286 18390 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:37:14 dist-test-slave-mn4r krb5kdc[17758](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502234, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:37:14.544685 18390 init.cc:377] Logged in from keytab as kudu/127.7.132.190@KRBTEST.COM (short username kudu)
I20260812 02:37:14.546326 18390 webserver.cc:460] Webserver started at http://127.7.132.190:36257/ using document root <none> and password file <none>
I20260812 02:37:14.547053 18390 fs_manager.cc:359] Metadata directory not provided
I20260812 02:37:14.547220 18390 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:37:14.552847 18390 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 02:37:14.555835 18405 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:37:14.557152 18390 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 02:37:14.557335 18390 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/master-0/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/master-0/wal
uuid: "4192d9eb1ca44723996b9cee0b1ec577"
format_stamp: "Formatted at 2026-08-12 02:37:11 on dist-test-slave-mn4r"
server_key: "6da6408bd5bde6de8f1e871bd6a54b9d"
server_key_iv: "cd71c0360550febcb87ebef5794b134d"
server_key_version: "encryptionkey@0"
I20260812 02:37:14.558368 18390 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:37:14.578320 18390 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:37:14.582458 18390 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:37:14.582743 18390 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:37:14.601991 18390 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.190:37037
I20260812 02:37:14.602005 18466 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.190:37037 every 8 connection(s)
I20260812 02:37:14.603613 18390 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
I20260812 02:37:14.605541 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 18390
I20260812 02:37:14.606401 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.129:39069
--local_ip_for_outbound_sockets=127.7.132.129
--tserver_master_addrs=127.7.132.190:37037
--webserver_port=34249
--webserver_interface=127.7.132.129
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.129.keytab
--principal=kudu/127.7.132.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:35499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.7.132.129
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/krb5kdc/oryx_127.7.132.129.keytab with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 02:37:14.610913 18467 sys_catalog.cc:263] Verifying existing consensus state
I20260812 02:37:14.614212 18467 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4192d9eb1ca44723996b9cee0b1ec577: Bootstrap starting.
I20260812 02:37:14.641111 18467 log.cc:824] T 00000000000000000000000000000000 P 4192d9eb1ca44723996b9cee0b1ec577: Log is configured to *not* fsync() on all Append() calls
I20260812 02:37:14.653173 18467 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4192d9eb1ca44723996b9cee0b1ec577: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 02:37:14.653875 18467 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4192d9eb1ca44723996b9cee0b1ec577: Bootstrap complete.
I20260812 02:37:14.667400 18467 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4192d9eb1ca44723996b9cee0b1ec577 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4192d9eb1ca44723996b9cee0b1ec577" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 37037 } }
I20260812 02:37:14.668658 18467 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4192d9eb1ca44723996b9cee0b1ec577 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4192d9eb1ca44723996b9cee0b1ec577, State: Initialized, Role: FOLLOWER
I20260812 02:37:14.669466 18467 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4192d9eb1ca44723996b9cee0b1ec577 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4192d9eb1ca44723996b9cee0b1ec577" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 37037 } }
I20260812 02:37:14.669802 18467 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4192d9eb1ca44723996b9cee0b1ec577 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 02:37:14.669943 18467 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4192d9eb1ca44723996b9cee0b1ec577 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 02:37:14.670163 18467 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4192d9eb1ca44723996b9cee0b1ec577 [term 1 FOLLOWER]: Advancing to term 2
I20260812 02:37:14.676422 18467 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4192d9eb1ca44723996b9cee0b1ec577 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4192d9eb1ca44723996b9cee0b1ec577" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 37037 } }
I20260812 02:37:14.677093 18467 leader_election.cc:302] T 00000000000000000000000000000000 P 4192d9eb1ca44723996b9cee0b1ec577 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4192d9eb1ca44723996b9cee0b1ec577; no voters:
I20260812 02:37:14.677814 18467 leader_election.cc:288] T 00000000000000000000000000000000 P 4192d9eb1ca44723996b9cee0b1ec577 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 02:37:14.677994 18472 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4192d9eb1ca44723996b9cee0b1ec577 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 02:37:14.679569 18472 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4192d9eb1ca44723996b9cee0b1ec577 [term 2 LEADER]: Becoming Leader. State: Replica: 4192d9eb1ca44723996b9cee0b1ec577, State: Running, Role: LEADER
I20260812 02:37:14.680230 18467 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4192d9eb1ca44723996b9cee0b1ec577 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:37:14.680156 18472 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4192d9eb1ca44723996b9cee0b1ec577 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4192d9eb1ca44723996b9cee0b1ec577" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 37037 } }
I20260812 02:37:14.682793 18474 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4192d9eb1ca44723996b9cee0b1ec577 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4192d9eb1ca44723996b9cee0b1ec577. Latest consensus state: current_term: 2 leader_uuid: "4192d9eb1ca44723996b9cee0b1ec577" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4192d9eb1ca44723996b9cee0b1ec577" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 37037 } } }
I20260812 02:37:14.683198 18474 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4192d9eb1ca44723996b9cee0b1ec577 [sys.catalog]: This master's current role is: LEADER
I20260812 02:37:14.683775 18473 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4192d9eb1ca44723996b9cee0b1ec577 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "4192d9eb1ca44723996b9cee0b1ec577" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4192d9eb1ca44723996b9cee0b1ec577" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 37037 } } }
I20260812 02:37:14.684182 18473 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4192d9eb1ca44723996b9cee0b1ec577 [sys.catalog]: This master's current role is: LEADER
I20260812 02:37:14.694896 18480 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:37:14.701094 18480 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 02:37:14.702579 18480 catalog_manager.cc:1259] Loaded cluster ID: dd86a88a7e65484cb3da1aced5b5e7c7
I20260812 02:37:14.702688 18480 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 02:37:14.706871 18480 catalog_manager.cc:1504] Loading token signing keys...
I20260812 02:37:14.708429 18480 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 4192d9eb1ca44723996b9cee0b1ec577: Loaded TSK: 0
I20260812 02:37:14.710058 18480 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 02:37:14.990664 18470 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:37:14.991004 18470 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:37:14.991158 18470 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:37:14.999002 18470 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:37:14.999171 18470 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:37:14.999349 18470 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.129
I20260812 02:37:15.009583 18470 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:35499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/krb5kdc/oryx_127.7.132.129.keytab
--principal=oryx/127.7.132.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.129:39069
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.132.129
--webserver_port=34249
--tserver_master_addrs=127.7.132.190:37037
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.129
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:37:15.011168 18470 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:37:15.012668 18470 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:37:15.028193 18501 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:37:15.028167 18504 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:37:15.028906 18500 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:37:15.029353 18470 server_base.cc:1034] running on GCE node
I20260812 02:37:15.029978 18470 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:37:15.031011 18470 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:37:15.032259 18470 hybrid_clock.cc:648] HybridClock initialized: now 1786502235032207 us; error 39 us; skew 500 ppm
I20260812 02:37:15.032629 18470 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:37:15 dist-test-slave-mn4r krb5kdc[17758](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502235, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.132.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:37:15.042011 18470 init.cc:377] Logged in from keytab as oryx/127.7.132.129@KRBTEST.COM (short username oryx)
I20260812 02:37:15.043790 18470 webserver.cc:460] Webserver started at http://127.7.132.129:34249/ using document root <none> and password file <none>
I20260812 02:37:15.044581 18470 fs_manager.cc:359] Metadata directory not provided
I20260812 02:37:15.044740 18470 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:37:15.050892 18470 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260812 02:37:15.053826 18513 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:37:15.055344 18470 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 02:37:15.055487 18470 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/ts-0/wal
uuid: "77cfaeb2c06b407db5b8e28ad5ad901d"
format_stamp: "Formatted at 2026-08-12 02:37:12 on dist-test-slave-mn4r"
server_key: "af8b67e8d29dad007cd68cb06a5ff150"
server_key_iv: "f68529c6fa6d14e03f046b2fef6e33d6"
server_key_version: "encryptionkey@0"
I20260812 02:37:15.056455 18470 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:37:15.074568 18470 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:37:15.078650 18470 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:37:15.078898 18470 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:37:15.079958 18470 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:37:15.081526 18470 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:37:15.081622 18470 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:37:15.081703 18470 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:37:15.081759 18470 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:37:15.114672 18470 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.129:39069
I20260812 02:37:15.114733 18649 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.129:39069 every 8 connection(s)
I20260812 02:37:15.116267 18470 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/ts-0/data/info.pb
I20260812 02:37:15.123962 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 18470
I20260812 02:37:15.124752 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.130:42401
--local_ip_for_outbound_sockets=127.7.132.130
--tserver_master_addrs=127.7.132.190:37037
--webserver_port=44897
--webserver_interface=127.7.132.130
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.130.keytab
--principal=kudu/127.7.132.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:35499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.7.132.130
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/krb5kdc/oryx_127.7.132.130.keytab with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 02:37:15 dist-test-slave-mn4r krb5kdc[17758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.129@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 02:37:15 dist-test-slave-mn4r krb5kdc[17758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.129@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
W20260812 02:37:15.487622 18657 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:37:15.488039 18657 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:37:15.488209 18657 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:37:15.495494 18657 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:37:15.495648 18657 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:37:15.495814 18657 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.130
I20260812 02:37:15.506135 18657 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:35499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/krb5kdc/oryx_127.7.132.130.keytab
--principal=oryx/127.7.132.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.130:42401
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.132.130
--webserver_port=44897
--tserver_master_addrs=127.7.132.190:37037
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.130
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:37:15.507627 18657 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:37:15.509135 18657 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:37:15.528344 18665 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:37:15.528420 18666 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:37:15.529510 18669 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:37:15.530303 18657 server_base.cc:1034] running on GCE node
I20260812 02:37:15.531489 18657 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:37:15.533201 18657 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:37:15.534518 18657 hybrid_clock.cc:648] HybridClock initialized: now 1786502235534416 us; error 63 us; skew 500 ppm
I20260812 02:37:15.535116 18657 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:37:15 dist-test-slave-mn4r krb5kdc[17758](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502235, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.132.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:37:15.540901 18657 init.cc:377] Logged in from keytab as oryx/127.7.132.130@KRBTEST.COM (short username oryx)
I20260812 02:37:15.543565 18657 webserver.cc:460] Webserver started at http://127.7.132.130:44897/ using document root <none> and password file <none>
I20260812 02:37:15.544921 18657 fs_manager.cc:359] Metadata directory not provided
I20260812 02:37:15.545169 18657 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:37:15.555531 18657 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.008s sys 0.000s
I20260812 02:37:15.560228 18676 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:37:15.562718 18657 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.004s
I20260812 02:37:15.562978 18657 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/ts-1/wal
uuid: "d8915b74441f419db7f678b24f1b6ccb"
format_stamp: "Formatted at 2026-08-12 02:37:12 on dist-test-slave-mn4r"
server_key: "e25b3f2d18c2d055d0503b5d7b0afed4"
server_key_iv: "3c29eda64a705bb532f0987d34b054a5"
server_key_version: "encryptionkey@0"
I20260812 02:37:15.565023 18657 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:37:15.619082 18657 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:37:15.624281 18657 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:37:15.624626 18657 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:37:15.625764 18657 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:37:15.627293 18657 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:37:15.627441 18657 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:37:15.627580 18657 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:37:15.627666 18657 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:37:15.676708 18657 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.130:42401
I20260812 02:37:15.676766 18809 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.130:42401 every 8 connection(s)
I20260812 02:37:15.678326 18657 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/ts-1/data/info.pb
I20260812 02:37:15.682554 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 18657
I20260812 02:37:15.683291 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.131:44117
--local_ip_for_outbound_sockets=127.7.132.131
--tserver_master_addrs=127.7.132.190:37037
--webserver_port=40203
--webserver_interface=127.7.132.131
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.131.keytab
--principal=kudu/127.7.132.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:35499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.7.132.131
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/krb5kdc/oryx_127.7.132.131.keytab with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 02:37:15 dist-test-slave-mn4r krb5kdc[17758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.130@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 02:37:15 dist-test-slave-mn4r krb5kdc[17758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.130@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
W20260812 02:37:16.350394 18814 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:37:16.350672 18814 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:37:16.350826 18814 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:37:16.358244 18814 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:37:16.358378 18814 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:37:16.358578 18814 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.131
I20260812 02:37:16.368646 18814 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:35499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/krb5kdc/oryx_127.7.132.131.keytab
--principal=oryx/127.7.132.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.131:44117
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.132.131
--webserver_port=40203
--tserver_master_addrs=127.7.132.190:37037
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.131
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:37:16.370023 18814 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:37:16.371517 18814 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:37:16.388993 18821 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:37:16.389945 18820 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:37:16.390338 18824 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:37:16.395323 18814 server_base.cc:1034] running on GCE node
I20260812 02:37:16.396766 18814 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:37:16.398550 18814 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:37:16.400065 18814 hybrid_clock.cc:648] HybridClock initialized: now 1786502236399948 us; error 88 us; skew 500 ppm
I20260812 02:37:16.400686 18814 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:37:16 dist-test-slave-mn4r krb5kdc[17758](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502236, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.132.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:37:16.406586 18814 init.cc:377] Logged in from keytab as oryx/127.7.132.131@KRBTEST.COM (short username oryx)
I20260812 02:37:16.415369 18814 webserver.cc:460] Webserver started at http://127.7.132.131:40203/ using document root <none> and password file <none>
I20260812 02:37:16.416409 18814 fs_manager.cc:359] Metadata directory not provided
I20260812 02:37:16.416622 18814 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:37:16.426203 18814 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.007s sys 0.000s
I20260812 02:37:16.430935 18832 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:37:16.433113 18814 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.001s
I20260812 02:37:16.433300 18814 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/ts-2/wal
uuid: "cf17c7e326334289a829888d92d40ce7"
format_stamp: "Formatted at 2026-08-12 02:37:13 on dist-test-slave-mn4r"
server_key: "71d9735807f6be7736346a50dd7088fe"
server_key_iv: "71dd8c7d9fe2f8401986aba64f97b4f2"
server_key_version: "encryptionkey@0"
I20260812 02:37:16.434737 18814 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:37:16.457659 18814 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:37:16.464263 18814 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:37:16.464588 18814 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:37:16.466022 18814 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
W20260812 02:37:16.467000 18655 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.334s user 0.003s sys 0.014s
I20260812 02:37:16.468129 18814 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:37:16.468242 18814 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:37:16.468343 18814 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:37:16.468439 18814 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:37:16.469398 18655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:15.119614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:37037 (local address 127.7.132.129:35931)
0812 02:37:15.120667 (+ 1053us) negotiation.cc:107] Waiting for socket to connect
0812 02:37:15.120743 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0812 02:37:15.123474 (+ 2731us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:37:15.125540 (+ 2066us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:37:15.125548 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:37:15.126185 (+ 637us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:37:15.126754 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:15.126774 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:15.129198 (+ 2424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:15.129202 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:15.130528 (+ 1326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:15.130541 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:15.132203 (+ 1662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:15.132209 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:15.132424 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:15.133141 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:37:15.133164 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:37:16.467990 (+1334826us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.132.190:37037: Server oryx/127.7.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":638,"thread_start_us":351,"threads_started":1}
W20260812 02:37:16.469998 18654 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:37:15.119575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:35931 (local address 127.7.132.190:37037)
0812 02:37:15.122540 (+ 2965us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:15.122552 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:15.122599 (+ 47us) server_negotiation.cc:407] Connection header received
0812 02:37:15.124406 (+ 1807us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:15.124430 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:15.124972 (+ 542us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:15.125419 (+ 447us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:37:15.126975 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:15.129007 (+ 2032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:15.130682 (+ 1675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:15.131960 (+ 1278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:15.132179 (+ 219us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:16.468677 (+1336498us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.132.129:35931: BlockingRecv error: recv got EOF from 127.7.132.129:35931 (error 108)
Metrics: {"server-negotiator.queue_time_us":2741,"thread_start_us":2560,"threads_started":1}
W20260812 02:37:16.470506 18651 heartbeater.cc:644] Failed to heartbeat to 127.7.132.190:37037 (0 consecutive failures): Not authorized: Failed to ping master at 127.7.132.190:37037: Client connection negotiation failed: client connection to 127.7.132.190:37037: Server oryx/127.7.132.190@KRBTEST.COM not found in Kerberos database
Aug 12 02:37:16 dist-test-slave-mn4r krb5kdc[17758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.129@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 02:37:16 dist-test-slave-mn4r krb5kdc[17758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.129@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
W20260812 02:37:16.486354 18654 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:37:16.471342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:51369 (local address 127.7.132.190:37037)
0812 02:37:16.471927 (+ 585us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:16.471933 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:16.471951 (+ 18us) server_negotiation.cc:407] Connection header received
0812 02:37:16.472018 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:16.472026 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:16.472113 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:16.472216 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:37:16.473429 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:16.474648 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:16.477397 (+ 2749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:16.478332 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:16.478489 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:16.485956 (+ 7467us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.132.129:51369: BlockingRecv error: recv got EOF from 127.7.132.129:51369 (error 108)
Metrics: {"server-negotiator.queue_time_us":440}
I20260812 02:37:16.487257 18655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:16.471281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:37037 (local address 127.7.132.129:51369)
0812 02:37:16.471535 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0812 02:37:16.471556 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 02:37:16.471679 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:37:16.472437 (+ 758us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:37:16.472442 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:37:16.472969 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:37:16.473205 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:16.473219 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:16.474881 (+ 1662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:16.474908 (+ 27us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:16.477107 (+ 2199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:16.477133 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:16.478838 (+ 1705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:16.478847 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:16.479128 (+ 281us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:16.480082 (+ 954us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:37:16.480107 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:37:16.486790 (+ 6683us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.132.190:37037: Server oryx/127.7.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":122}
Aug 12 02:37:16 dist-test-slave-mn4r krb5kdc[17758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.129@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 02:37:16 dist-test-slave-mn4r krb5kdc[17758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.129@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 02:37:16.504195 18655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:16.488217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:37037 (local address 127.7.132.129:34443)
0812 02:37:16.488573 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0812 02:37:16.488606 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0812 02:37:16.488781 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:37:16.490027 (+ 1246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:37:16.490036 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:37:16.490583 (+ 547us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:37:16.490857 (+ 274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:16.490875 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:16.492741 (+ 1866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:16.492768 (+ 27us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:16.494606 (+ 1838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:16.494630 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:16.496096 (+ 1466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:16.496107 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:16.496440 (+ 333us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:16.497395 (+ 955us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:37:16.497434 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:37:16.503714 (+ 6280us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.132.190:37037: Server oryx/127.7.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":177}
W20260812 02:37:16.504454 18654 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:37:16.489211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:34443 (local address 127.7.132.190:37037)
0812 02:37:16.489413 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:16.489421 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:16.489444 (+ 23us) server_negotiation.cc:407] Connection header received
0812 02:37:16.489526 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:16.489534 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:16.489658 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:16.489777 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:37:16.491081 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:16.492507 (+ 1426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:16.494842 (+ 2335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:16.495699 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:16.495865 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:16.504092 (+ 8227us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.132.129:34443: BlockingRecv error: recv got EOF from 127.7.132.129:34443 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
W20260812 02:37:16.504752 18651 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 02:37:16.532821 18814 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.131:44117
I20260812 02:37:16.532863 18974 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.131:44117 every 8 connection(s)
I20260812 02:37:16.534437 18814 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786502156971023-7698-0/minicluster-data/ts-2/data/info.pb
I20260812 02:37:16.536192 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 18814
Aug 12 02:37:16 dist-test-slave-mn4r krb5kdc[17758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.131@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 02:37:16 dist-test-slave-mn4r krb5kdc[17758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.131@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 02:37:16.561668 18654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:16.537932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36196 (local address 127.7.132.190:37037)
0812 02:37:16.538169 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:16.538175 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:16.538634 (+ 459us) server_negotiation.cc:407] Connection header received
0812 02:37:16.538854 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:16.538863 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:16.538988 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:16.539152 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:37:16.542447 (+ 3295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:16.544126 (+ 1679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:16.546758 (+ 2632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:16.547783 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:16.547974 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:16.552584 (+ 4610us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:37:16.552595 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:37:16.552615 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:37:16.552666 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:37:16.556784 (+ 4118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:37:16.558054 (+ 1270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:37:16.558061 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:37:16.558067 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:37:16.558343 (+ 276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:37:16.558993 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:37:16.559000 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:37:16.559003 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:37:16.559667 (+ 664us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:37:16.560025 (+ 358us) server_negotiation.cc:1035] Waiting for connection context
0812 02:37:16.560809 (+ 784us) server_negotiation.cc:299] Negotiation successful
0812 02:37:16.561240 (+ 431us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
W20260812 02:37:16.832453 18813 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.136s user 0.001s sys 0.012s
W20260812 02:37:16.834431 18812 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:37:15.681461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:44135 (local address 127.7.132.190:37037)
0812 02:37:15.682033 (+ 572us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:15.682037 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:15.686879 (+ 4842us) server_negotiation.cc:407] Connection header received
0812 02:37:15.686961 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:15.686968 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:15.687095 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:15.687214 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:37:15.690384 (+ 3170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:15.692226 (+ 1842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:15.694532 (+ 2306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:15.695039 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:15.695175 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:16.833999 (+1138824us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.132.130:44135: BlockingRecv error: recv got EOF from 127.7.132.130:44135 (error 108)
Metrics: {"server-negotiator.queue_time_us":376,"thread_start_us":276,"threads_started":1}
I20260812 02:37:16.835510 18813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:15.682539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:37037 (local address 127.7.132.130:44135)
0812 02:37:15.683596 (+ 1057us) negotiation.cc:107] Waiting for socket to connect
0812 02:37:15.683649 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0812 02:37:15.686492 (+ 2843us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:37:15.687687 (+ 1195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:37:15.687707 (+ 20us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:37:15.688807 (+ 1100us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:37:15.689978 (+ 1171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:15.690013 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:15.692480 (+ 2467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:15.692488 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:15.694319 (+ 1831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:15.694350 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:15.695542 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:15.695547 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:15.695801 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:15.696585 (+ 784us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:37:15.696614 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:37:16.833805 (+1137191us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.132.190:37037: Server oryx/127.7.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":692,"thread_start_us":414,"threads_started":1}
W20260812 02:37:16.836510 18810 heartbeater.cc:644] Failed to heartbeat to 127.7.132.190:37037 (0 consecutive failures): Not authorized: Failed to ping master at 127.7.132.190:37037: Client connection negotiation failed: client connection to 127.7.132.190:37037: Server oryx/127.7.132.190@KRBTEST.COM not found in Kerberos database
Aug 12 02:37:16 dist-test-slave-mn4r krb5kdc[17758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.130@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 02:37:16 dist-test-slave-mn4r krb5kdc[17758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.130@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 02:37:16.847055 18813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:16.837209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:37037 (local address 127.7.132.130:47873)
0812 02:37:16.837410 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0812 02:37:16.837434 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0812 02:37:16.837556 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:37:16.837886 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:37:16.837892 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:37:16.838348 (+ 456us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:37:16.838522 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:16.838531 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:16.839774 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:16.839780 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:16.840830 (+ 1050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:16.840842 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:16.841607 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:16.841613 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:16.841790 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:16.842486 (+ 696us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:37:16.842504 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:37:16.846821 (+ 4317us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.132.190:37037: Server oryx/127.7.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":67}
W20260812 02:37:16.847532 18812 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:37:16.837332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:47873 (local address 127.7.132.190:37037)
0812 02:37:16.837535 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:16.837540 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:16.837559 (+ 19us) server_negotiation.cc:407] Connection header received
0812 02:37:16.837673 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:16.837679 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:16.837754 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:16.837855 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:37:16.838669 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:16.839624 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:16.841007 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:16.841426 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:16.841567 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:16.847177 (+ 5610us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.132.130:47873: BlockingRecv error: recv got EOF from 127.7.132.130:47873 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
Aug 12 02:37:16 dist-test-slave-mn4r krb5kdc[17758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.130@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 02:37:16 dist-test-slave-mn4r krb5kdc[17758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.130@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 02:37:16.858520 18813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:16.847727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:37037 (local address 127.7.132.130:53091)
0812 02:37:16.847864 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0812 02:37:16.847883 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 02:37:16.847986 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:37:16.848762 (+ 776us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:37:16.848765 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:37:16.849047 (+ 282us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:37:16.849197 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:16.849204 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:16.851237 (+ 2033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:16.851241 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:16.852181 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:16.852192 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:16.853594 (+ 1402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:16.853599 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:16.853745 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:16.854422 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:37:16.854439 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:37:16.858228 (+ 3789us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.132.190:37037: Server oryx/127.7.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":35}
W20260812 02:37:16.858749 18812 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:37:16.848136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:53091 (local address 127.7.132.190:37037)
0812 02:37:16.848348 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:16.848358 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:16.848383 (+ 25us) server_negotiation.cc:407] Connection header received
0812 02:37:16.848466 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:16.848473 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:16.848595 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:16.848744 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:37:16.849351 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:16.851060 (+ 1709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:16.852334 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:16.853388 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:16.853578 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:16.858446 (+ 4868us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.132.130:53091: BlockingRecv error: recv got EOF from 127.7.132.130:53091 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260812 02:37:16.859051 18810 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 12 02:37:17 dist-test-slave-mn4r krb5kdc[17758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.129@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 02:37:17 dist-test-slave-mn4r krb5kdc[17758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.129@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 02:37:17.516060 18981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:17.505470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:37037 (local address 127.7.132.129:51643)
0812 02:37:17.506163 (+ 693us) negotiation.cc:107] Waiting for socket to connect
0812 02:37:17.506195 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0812 02:37:17.506353 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:37:17.506781 (+ 428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:37:17.506786 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:37:17.507280 (+ 494us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:37:17.507438 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:17.507447 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:17.508553 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:17.508559 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:17.509537 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:17.509549 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:17.510410 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:17.510416 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:17.510607 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:17.511272 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:37:17.511291 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:37:17.515813 (+ 4522us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.132.190:37037: Server oryx/127.7.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":505,"thread_start_us":368,"threads_started":1}
W20260812 02:37:17.516706 18982 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:37:17.505750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:51643 (local address 127.7.132.190:37037)
0812 02:37:17.506459 (+ 709us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:17.506466 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:17.506491 (+ 25us) server_negotiation.cc:407] Connection header received
0812 02:37:17.506541 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:17.506544 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:17.506603 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:17.506729 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:37:17.507615 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:17.508395 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:17.509708 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:17.510215 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:17.510356 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:17.516240 (+ 5884us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.132.129:51643: BlockingRecv error: recv got EOF from 127.7.132.129:51643 (error 108)
Metrics: {"server-negotiator.queue_time_us":558,"thread_start_us":396,"threads_started":1}
W20260812 02:37:17.712172 18978 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.154s user 0.009s sys 0.006s
W20260812 02:37:17.713583 18979 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:37:16.539415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:37533 (local address 127.7.132.190:37037)
0812 02:37:16.540275 (+ 860us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:16.540285 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:16.548426 (+ 8141us) server_negotiation.cc:407] Connection header received
0812 02:37:16.550324 (+ 1898us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:16.550329 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:16.550442 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:16.550565 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:37:16.552231 (+ 1666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:16.553539 (+ 1308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:16.556081 (+ 2542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:16.557111 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:16.557301 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:17.713248 (+1155947us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.132.131:37533: BlockingRecv error: recv got EOF from 127.7.132.131:37533 (error 108)
Metrics: {"server-negotiator.queue_time_us":567,"thread_start_us":374,"threads_started":1}
I20260812 02:37:17.714259 18978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:16.538890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:37037 (local address 127.7.132.131:37533)
0812 02:37:16.547382 (+ 8492us) negotiation.cc:107] Waiting for socket to connect
0812 02:37:16.547445 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0812 02:37:16.549988 (+ 2543us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:37:16.550925 (+ 937us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:37:16.550935 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:37:16.551461 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:37:16.552048 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:16.552066 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:16.553707 (+ 1641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:16.553710 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:16.555850 (+ 2140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:16.555890 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:16.557403 (+ 1513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:16.557411 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:16.557715 (+ 304us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:16.558528 (+ 813us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:37:16.558560 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:37:17.713054 (+1154494us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.132.190:37037: Server oryx/127.7.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":8173,"thread_start_us":7978,"threads_started":1}
W20260812 02:37:17.715113 18975 heartbeater.cc:644] Failed to heartbeat to 127.7.132.190:37037 (0 consecutive failures): Not authorized: Failed to ping master at 127.7.132.190:37037: Client connection negotiation failed: client connection to 127.7.132.190:37037: Server oryx/127.7.132.190@KRBTEST.COM not found in Kerberos database
Aug 12 02:37:17 dist-test-slave-mn4r krb5kdc[17758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.131@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 02:37:17 dist-test-slave-mn4r krb5kdc[17758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.131@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 02:37:17.725162 18978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:17.715797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:37037 (local address 127.7.132.131:42047)
0812 02:37:17.715974 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0812 02:37:17.715994 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 02:37:17.716140 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:37:17.716483 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:37:17.716489 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:37:17.716886 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:37:17.717054 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:17.717063 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:17.718154 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:17.718158 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:17.719129 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:17.719140 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:17.719854 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:17.719863 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:17.720024 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:17.720703 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:37:17.720720 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:37:17.724841 (+ 4121us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.132.190:37037: Server oryx/127.7.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":61}
W20260812 02:37:17.725323 18979 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:37:17.715889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:42047 (local address 127.7.132.190:37037)
0812 02:37:17.716196 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:17.716203 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:17.716222 (+ 19us) server_negotiation.cc:407] Connection header received
0812 02:37:17.716283 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:17.716287 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:17.716342 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:17.716444 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:37:17.717195 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:17.717967 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:17.719271 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:17.719668 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:17.719797 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:17.725050 (+ 5253us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.132.131:42047: BlockingRecv error: recv got EOF from 127.7.132.131:42047 (error 108)
Metrics: {"server-negotiator.queue_time_us":177}
Aug 12 02:37:17 dist-test-slave-mn4r krb5kdc[17758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.131@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 02:37:17 dist-test-slave-mn4r krb5kdc[17758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.131@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 02:37:17.734244 18978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:17.725989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:37037 (local address 127.7.132.131:43213)
0812 02:37:17.726198 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0812 02:37:17.726224 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0812 02:37:17.726306 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:37:17.726578 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:37:17.726581 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:37:17.726885 (+ 304us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:37:17.727036 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:17.727044 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:17.728056 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:17.728060 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:17.729022 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:17.729034 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:17.729745 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:17.729750 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:17.729878 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:17.730522 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:37:17.730541 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:37:17.733889 (+ 3348us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.132.190:37037: Server oryx/127.7.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":59}
W20260812 02:37:17.734421 18979 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:37:17.726146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:43213 (local address 127.7.132.190:37037)
0812 02:37:17.726324 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:17.726328 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:17.726345 (+ 17us) server_negotiation.cc:407] Connection header received
0812 02:37:17.726392 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:17.726395 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:17.726450 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:17.726530 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:37:17.727173 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:17.727920 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:17.729165 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:17.729608 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:17.729707 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:17.734143 (+ 4436us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.132.131:43213: BlockingRecv error: recv got EOF from 127.7.132.131:43213 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260812 02:37:17.734591 18975 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 12 02:37:17 dist-test-slave-mn4r krb5kdc[17758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.130@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 02:37:17 dist-test-slave-mn4r krb5kdc[17758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.130@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 02:37:17.875747 18983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:17.859823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:37037 (local address 127.7.132.130:47583)
0812 02:37:17.860696 (+ 873us) negotiation.cc:107] Waiting for socket to connect
0812 02:37:17.860714 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 02:37:17.860838 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:37:17.861394 (+ 556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:37:17.861402 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:37:17.861962 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:37:17.862152 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:17.862162 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:17.863392 (+ 1230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:17.863396 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:17.864366 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:17.864378 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:17.865214 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:17.865218 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:17.865341 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:17.871222 (+ 5881us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:37:17.871244 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:37:17.875500 (+ 4256us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.132.190:37037: Server oryx/127.7.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":750,"thread_start_us":617,"threads_started":1}
W20260812 02:37:17.876025 18979 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:37:17.859989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:47583 (local address 127.7.132.190:37037)
0812 02:37:17.860212 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:17.860280 (+ 68us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:17.860810 (+ 530us) server_negotiation.cc:407] Connection header received
0812 02:37:17.861040 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:17.861044 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:17.861193 (+ 149us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:17.861355 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:37:17.862305 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:17.863233 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:17.864545 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:17.865038 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:17.865148 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:17.875743 (+ 10595us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.132.130:47583: BlockingRecv error: recv got EOF from 127.7.132.130:47583 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Aug 12 02:37:18 dist-test-slave-mn4r krb5kdc[17758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.129@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 02:37:18 dist-test-slave-mn4r krb5kdc[17758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.129@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 02:37:18.527396 18984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:18.517330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:37037 (local address 127.7.132.129:55741)
0812 02:37:18.518253 (+ 923us) negotiation.cc:107] Waiting for socket to connect
0812 02:37:18.518314 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0812 02:37:18.518493 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:37:18.518842 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:37:18.518845 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:37:18.519296 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:37:18.519485 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:18.519495 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:18.520654 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:18.520658 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:18.521582 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:18.521593 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:18.522386 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:18.522391 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:18.522538 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:18.523176 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:37:18.523194 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:37:18.527113 (+ 3919us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.132.190:37037: Server oryx/127.7.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":318,"threads_started":1}
W20260812 02:37:18.527611 18985 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:37:18.517505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:55741 (local address 127.7.132.190:37037)
0812 02:37:18.518308 (+ 803us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:18.518315 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:18.518453 (+ 138us) server_negotiation.cc:407] Connection header received
0812 02:37:18.518627 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:18.518631 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:18.518693 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:18.518784 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:37:18.519658 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:18.520497 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:18.521728 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:18.522228 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:18.522334 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:18.527336 (+ 5002us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.132.129:55741: BlockingRecv error: recv got EOF from 127.7.132.129:55741 (error 108)
Metrics: {"server-negotiator.queue_time_us":668,"thread_start_us":467,"threads_started":1}
Aug 12 02:37:18 dist-test-slave-mn4r krb5kdc[17758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.131@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 02:37:18 dist-test-slave-mn4r krb5kdc[17758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.131@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 02:37:18.744767 18986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:18.735203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:37037 (local address 127.7.132.131:36847)
0812 02:37:18.735804 (+ 601us) negotiation.cc:107] Waiting for socket to connect
0812 02:37:18.735821 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 02:37:18.735944 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:37:18.736255 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:37:18.736258 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:37:18.736675 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:37:18.736824 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:18.736833 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:18.737867 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:18.737870 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:18.739045 (+ 1175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:18.739060 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:18.740084 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:18.740091 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:18.740262 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:18.740967 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:37:18.741025 (+ 58us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:37:18.744513 (+ 3488us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.132.190:37037: Server oryx/127.7.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":353,"threads_started":1}
W20260812 02:37:18.744952 18985 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:37:18.735279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:36847 (local address 127.7.132.190:37037)
0812 02:37:18.735507 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:18.735512 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:18.735905 (+ 393us) server_negotiation.cc:407] Connection header received
0812 02:37:18.736058 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:18.736062 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:18.736124 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:18.736211 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:37:18.736955 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:18.737739 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:18.739189 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:18.739834 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:18.740002 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:18.744715 (+ 4713us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.132.131:36847: BlockingRecv error: recv got EOF from 127.7.132.131:36847 (error 108)
Metrics: {"server-negotiator.queue_time_us":91}
Aug 12 02:37:18 dist-test-slave-mn4r krb5kdc[17758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.130@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 02:37:18 dist-test-slave-mn4r krb5kdc[17758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.130@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 02:37:18.886691 18987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:18.876809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:37037 (local address 127.7.132.130:47795)
0812 02:37:18.877425 (+ 616us) negotiation.cc:107] Waiting for socket to connect
0812 02:37:18.877442 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 02:37:18.877563 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:37:18.877943 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:37:18.877948 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:37:18.878470 (+ 522us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:37:18.878664 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:18.878674 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:18.879753 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:18.879757 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:18.880775 (+ 1018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:18.880788 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:18.881706 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:18.881712 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:18.881854 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:18.882546 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:37:18.882567 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:37:18.886400 (+ 3833us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.132.190:37037: Server oryx/127.7.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":359,"threads_started":1}
W20260812 02:37:18.886847 18985 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:37:18.876930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:47795 (local address 127.7.132.190:37037)
0812 02:37:18.877160 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:18.877167 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:18.877527 (+ 360us) server_negotiation.cc:407] Connection header received
0812 02:37:18.877691 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:18.877696 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:18.877771 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:18.877854 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:37:18.878814 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:18.879608 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:18.880969 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:18.881515 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:18.881612 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:18.886624 (+ 5012us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.132.130:47795: BlockingRecv error: recv got EOF from 127.7.132.130:47795 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Aug 12 02:37:19 dist-test-slave-mn4r krb5kdc[17758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.129@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 02:37:19 dist-test-slave-mn4r krb5kdc[17758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.129@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 02:37:19.538843 18989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:19.528521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:37037 (local address 127.7.132.129:50121)
0812 02:37:19.529088 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0812 02:37:19.529109 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 02:37:19.529219 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:37:19.529728 (+ 509us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:37:19.529731 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:37:19.530240 (+ 509us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:37:19.530432 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:19.530443 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:19.531477 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:19.531483 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:19.533108 (+ 1625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:19.533126 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:19.534141 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:19.534147 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:19.534347 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:19.535011 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:37:19.535030 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:37:19.538609 (+ 3579us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.132.190:37037: Server oryx/127.7.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":259,"threads_started":1}
W20260812 02:37:19.539228 18990 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:37:19.528710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:50121 (local address 127.7.132.190:37037)
0812 02:37:19.529431 (+ 721us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:19.529436 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:19.529452 (+ 16us) server_negotiation.cc:407] Connection header received
0812 02:37:19.529499 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:19.529502 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:19.529571 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:19.529663 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:37:19.530588 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:19.531336 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:19.533246 (+ 1910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:19.533957 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:19.534121 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:19.538813 (+ 4692us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.132.129:50121: BlockingRecv error: recv got EOF from 127.7.132.129:50121 (error 108)
Metrics: {"server-negotiator.queue_time_us":609,"thread_start_us":471,"threads_started":1}
Aug 12 02:37:19 dist-test-slave-mn4r krb5kdc[17758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.131@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 02:37:19 dist-test-slave-mn4r krb5kdc[17758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.131@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 02:37:19.755738 18991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:19.745695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:37037 (local address 127.7.132.131:44403)
0812 02:37:19.746250 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0812 02:37:19.746269 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 02:37:19.746399 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:37:19.746720 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:37:19.746723 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:37:19.747199 (+ 476us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:37:19.747401 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:19.747410 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:19.748508 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:19.748513 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:19.749551 (+ 1038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:19.749564 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:19.750339 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:19.750344 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:19.750483 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:19.751286 (+ 803us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:37:19.751312 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:37:19.755355 (+ 4043us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.132.190:37037: Server oryx/127.7.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":294,"threads_started":1}
W20260812 02:37:19.755779 18990 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:37:19.745787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:44403 (local address 127.7.132.190:37037)
0812 02:37:19.746066 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:19.746073 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:19.746373 (+ 300us) server_negotiation.cc:407] Connection header received
0812 02:37:19.746516 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:19.746520 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:19.746586 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:19.746673 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:37:19.747558 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:19.748358 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:19.749714 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:19.750182 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:19.750279 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:19.755566 (+ 5287us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.132.131:44403: BlockingRecv error: recv got EOF from 127.7.132.131:44403 (error 108)
Metrics: {"server-negotiator.queue_time_us":118}
Aug 12 02:37:19 dist-test-slave-mn4r krb5kdc[17758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.130@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 02:37:19 dist-test-slave-mn4r krb5kdc[17758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.130@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 02:37:19.897744 18992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:19.887654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:37037 (local address 127.7.132.130:50303)
0812 02:37:19.888229 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0812 02:37:19.888245 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 02:37:19.888364 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:37:19.888769 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:37:19.888773 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:37:19.889157 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:37:19.889324 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:19.889333 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:19.890742 (+ 1409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:19.890746 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:19.891733 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:19.891746 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:19.892593 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:19.892598 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:19.892749 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:19.893436 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:37:19.893458 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:37:19.897473 (+ 4015us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.132.190:37037: Server oryx/127.7.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":315,"threads_started":1}
W20260812 02:37:19.898125 18990 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:37:19.887767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:50303 (local address 127.7.132.190:37037)
0812 02:37:19.887989 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:19.887994 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:19.888318 (+ 324us) server_negotiation.cc:407] Connection header received
0812 02:37:19.888487 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:19.888491 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:19.888568 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:19.888653 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:37:19.889612 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:19.890576 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:19.891894 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:19.892427 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:19.892536 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:19.897677 (+ 5141us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.132.130:50303: BlockingRecv error: recv got EOF from 127.7.132.130:50303 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 12 02:37:20 dist-test-slave-mn4r krb5kdc[17758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.129@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 02:37:20 dist-test-slave-mn4r krb5kdc[17758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.129@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 02:37:20.554917 18993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:20.539831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:37037 (local address 127.7.132.129:49851)
0812 02:37:20.540487 (+ 656us) negotiation.cc:107] Waiting for socket to connect
0812 02:37:20.540517 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0812 02:37:20.540663 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:37:20.541054 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:37:20.541059 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:37:20.541477 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:37:20.541648 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:20.541658 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:20.542892 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:20.542902 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:20.544087 (+ 1185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:20.544100 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:20.544831 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:20.544837 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:20.545040 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:20.550447 (+ 5407us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:37:20.550467 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:37:20.554544 (+ 4077us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.132.190:37037: Server oryx/127.7.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":505,"thread_start_us":273,"threads_started":1}
W20260812 02:37:20.555166 18995 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:37:20.539916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:49851 (local address 127.7.132.190:37037)
0812 02:37:20.540669 (+ 753us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:20.540675 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:20.540694 (+ 19us) server_negotiation.cc:407] Connection header received
0812 02:37:20.540767 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:20.540773 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:20.540851 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:20.540970 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:37:20.541812 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:20.542724 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:20.544236 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:20.544681 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:20.544816 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:20.554806 (+ 9990us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.132.129:49851: BlockingRecv error: recv got EOF from 127.7.132.129:49851 (error 108)
Metrics: {"server-negotiator.queue_time_us":644,"thread_start_us":444,"threads_started":1}
Aug 12 02:37:20 dist-test-slave-mn4r krb5kdc[17758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.131@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 02:37:20 dist-test-slave-mn4r krb5kdc[17758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.131@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 02:37:20.766404 18996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:20.756771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:37037 (local address 127.7.132.131:42637)
0812 02:37:20.757337 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0812 02:37:20.757352 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 02:37:20.757533 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:37:20.757971 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:37:20.757977 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:37:20.758430 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:37:20.758609 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:20.758619 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:20.759646 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:20.759652 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:20.760688 (+ 1036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:20.760700 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:20.761421 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:20.761426 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:20.761630 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:20.762352 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:37:20.762371 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:37:20.766036 (+ 3665us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.132.190:37037: Server oryx/127.7.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":288,"threads_started":1}
W20260812 02:37:20.766528 18995 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:37:20.756896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:42637 (local address 127.7.132.190:37037)
0812 02:37:20.757244 (+ 348us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:20.757251 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:20.757460 (+ 209us) server_negotiation.cc:407] Connection header received
0812 02:37:20.757677 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:20.757682 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:20.757755 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:20.757862 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:37:20.758760 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:20.759505 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:20.760832 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:20.761261 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:20.761419 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:20.766290 (+ 4871us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.132.131:42637: BlockingRecv error: recv got EOF from 127.7.132.131:42637 (error 108)
Metrics: {"server-negotiator.queue_time_us":201}
Aug 12 02:37:20 dist-test-slave-mn4r krb5kdc[17758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.130@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 02:37:20 dist-test-slave-mn4r krb5kdc[17758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.130@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 02:37:20.909049 18998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:20.898777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:37037 (local address 127.7.132.130:54725)
0812 02:37:20.899352 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0812 02:37:20.899369 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 02:37:20.899488 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:37:20.899900 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:37:20.899904 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:37:20.900310 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:37:20.900481 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:20.900490 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:20.901728 (+ 1238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:20.901732 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:20.902991 (+ 1259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:20.903006 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:20.903820 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:20.903825 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:20.904017 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:20.904668 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:37:20.904687 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:37:20.908694 (+ 4007us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.132.190:37037: Server oryx/127.7.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":334,"threads_started":1}
W20260812 02:37:20.909452 18995 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:37:20.898865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:54725 (local address 127.7.132.190:37037)
0812 02:37:20.899050 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:20.899054 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:20.899465 (+ 411us) server_negotiation.cc:407] Connection header received
0812 02:37:20.899655 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:20.899660 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:20.899747 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:20.899845 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:37:20.900628 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:20.901558 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:20.903149 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:20.903662 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:20.903776 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:20.908974 (+ 5198us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.132.130:54725: BlockingRecv error: recv got EOF from 127.7.132.130:54725 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 12 02:37:21 dist-test-slave-mn4r krb5kdc[17758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.129@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 02:37:21 dist-test-slave-mn4r krb5kdc[17758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.129@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 02:37:21.566309 18999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:21.556130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:37037 (local address 127.7.132.129:49577)
0812 02:37:21.556676 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0812 02:37:21.556698 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 02:37:21.556808 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:37:21.557379 (+ 571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:37:21.557382 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:37:21.557820 (+ 438us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:37:21.557988 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:21.557997 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:21.559185 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:21.559190 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:21.560239 (+ 1049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:21.560257 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:21.560991 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:21.560997 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:21.561206 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:21.561859 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:37:21.561877 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:37:21.566010 (+ 4133us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.132.190:37037: Server oryx/127.7.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":262,"threads_started":1}
W20260812 02:37:21.566557 19000 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:37:21.556302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:49577 (local address 127.7.132.190:37037)
0812 02:37:21.557093 (+ 791us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:21.557098 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:21.557115 (+ 17us) server_negotiation.cc:407] Connection header received
0812 02:37:21.557163 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:21.557167 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:21.557227 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:21.557325 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:37:21.558218 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:21.559027 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:21.560382 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:21.560810 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:21.560978 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:21.566290 (+ 5312us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.132.129:49577: BlockingRecv error: recv got EOF from 127.7.132.129:49577 (error 108)
Metrics: {"server-negotiator.queue_time_us":692,"thread_start_us":509,"threads_started":1}
Aug 12 02:37:21 dist-test-slave-mn4r krb5kdc[17758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.131@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 02:37:21 dist-test-slave-mn4r krb5kdc[17758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.131@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 02:37:21.777531 19002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:21.767693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:37037 (local address 127.7.132.131:52887)
0812 02:37:21.768554 (+ 861us) negotiation.cc:107] Waiting for socket to connect
0812 02:37:21.768577 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0812 02:37:21.768744 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:37:21.769246 (+ 502us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:37:21.769251 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:37:21.769654 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:37:21.769852 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:21.769862 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:21.770965 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:21.770969 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:21.772074 (+ 1105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:21.772087 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:21.772865 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:21.772870 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:21.773075 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:21.773726 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:37:21.773744 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:37:21.777274 (+ 3530us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.132.190:37037: Server oryx/127.7.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":721,"thread_start_us":566,"threads_started":1}
W20260812 02:37:21.777721 19000 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:37:21.767795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:52887 (local address 127.7.132.190:37037)
0812 02:37:21.768108 (+ 313us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:21.768118 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:21.768689 (+ 571us) server_negotiation.cc:407] Connection header received
0812 02:37:21.768978 (+ 289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:21.768983 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:21.769057 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:21.769140 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:37:21.770006 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:21.770785 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:21.772230 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:21.772700 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:21.772820 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:21.777496 (+ 4676us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.132.131:52887: BlockingRecv error: recv got EOF from 127.7.132.131:52887 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 12 02:37:21 dist-test-slave-mn4r krb5kdc[17758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.130@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 02:37:21 dist-test-slave-mn4r krb5kdc[17758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.130@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 02:37:21.919818 19003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:21.910173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:37037 (local address 127.7.132.130:52299)
0812 02:37:21.911059 (+ 886us) negotiation.cc:107] Waiting for socket to connect
0812 02:37:21.911077 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 02:37:21.911254 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:37:21.911653 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:37:21.911659 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:37:21.912142 (+ 483us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:37:21.912305 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:21.912314 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:21.913376 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:21.913383 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:21.914370 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:21.914383 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:21.915139 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:21.915145 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:21.915346 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:21.915934 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:37:21.915999 (+ 65us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:37:21.919592 (+ 3593us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.132.190:37037: Server oryx/127.7.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":728,"thread_start_us":599,"threads_started":1}
W20260812 02:37:21.920042 19000 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:37:21.910409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:52299 (local address 127.7.132.190:37037)
0812 02:37:21.910657 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:21.910667 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:21.911178 (+ 511us) server_negotiation.cc:407] Connection header received
0812 02:37:21.911404 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:21.911410 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:21.911502 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:21.911613 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:37:21.912472 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:21.913236 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:21.914529 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:21.914984 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:21.915125 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:21.919791 (+ 4666us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.132.130:52299: BlockingRecv error: recv got EOF from 127.7.132.130:52299 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 12 02:37:22 dist-test-slave-mn4r krb5kdc[17758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.129@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 02:37:22 dist-test-slave-mn4r krb5kdc[17758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.129@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 02:37:22.577080 19004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:22.567353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:37037 (local address 127.7.132.129:34237)
0812 02:37:22.567934 (+ 581us) negotiation.cc:107] Waiting for socket to connect
0812 02:37:22.567965 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0812 02:37:22.568102 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:37:22.568450 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:37:22.568454 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:37:22.568902 (+ 448us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:37:22.569077 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:22.569086 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:22.570157 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:22.570162 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:22.571192 (+ 1030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:22.571205 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:22.571965 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:22.571970 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:22.572110 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:22.572804 (+ 694us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:37:22.572821 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:37:22.576804 (+ 3983us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.132.190:37037: Server oryx/127.7.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":335,"threads_started":1}
W20260812 02:37:22.577291 19005 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:37:22.567374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:34237 (local address 127.7.132.190:37037)
0812 02:37:22.567928 (+ 554us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:22.567934 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:22.568062 (+ 128us) server_negotiation.cc:407] Connection header received
0812 02:37:22.568206 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:22.568210 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:22.568278 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:22.568367 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:37:22.569220 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:22.569954 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:22.571344 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:22.571757 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:22.571864 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:22.577017 (+ 5153us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.132.129:34237: BlockingRecv error: recv got EOF from 127.7.132.129:34237 (error 108)
Metrics: {"server-negotiator.queue_time_us":433,"thread_start_us":296,"threads_started":1}
Aug 12 02:37:22 dist-test-slave-mn4r krb5kdc[17758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.131@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 02:37:22 dist-test-slave-mn4r krb5kdc[17758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.131@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 02:37:22.788669 19006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:22.778633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:37037 (local address 127.7.132.131:48727)
0812 02:37:22.779171 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0812 02:37:22.779187 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 02:37:22.779353 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:37:22.779718 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:37:22.779722 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:37:22.780299 (+ 577us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:37:22.780503 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:22.780513 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:22.781669 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:22.781674 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:22.782637 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:22.782648 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:22.783384 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:22.783390 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:22.783587 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:22.784206 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:37:22.784225 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:37:22.788406 (+ 4181us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.132.190:37037: Server oryx/127.7.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":435,"thread_start_us":276,"threads_started":1}
W20260812 02:37:22.788820 19005 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:37:22.778724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:48727 (local address 127.7.132.190:37037)
0812 02:37:22.778971 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:22.779016 (+ 45us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:22.779275 (+ 259us) server_negotiation.cc:407] Connection header received
0812 02:37:22.779489 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:22.779494 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:22.779572 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:22.779680 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:37:22.780650 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:22.781528 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:22.782787 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:22.783231 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:22.783368 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:22.788607 (+ 5239us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.132.131:48727: BlockingRecv error: recv got EOF from 127.7.132.131:48727 (error 108)
Metrics: {"server-negotiator.queue_time_us":89}
Aug 12 02:37:22 dist-test-slave-mn4r krb5kdc[17758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.130@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 02:37:22 dist-test-slave-mn4r krb5kdc[17758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.130@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 02:37:22.930378 19007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:22.920926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:37037 (local address 127.7.132.130:37243)
0812 02:37:22.921427 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0812 02:37:22.921445 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 02:37:22.921585 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:37:22.921945 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:37:22.921949 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:37:22.922413 (+ 464us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:37:22.922603 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:22.922613 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:22.923734 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:22.923738 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:22.924698 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:22.924711 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:22.925468 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:22.925472 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:22.925623 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:22.926328 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:37:22.926347 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:37:22.930138 (+ 3791us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.132.190:37037: Server oryx/127.7.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":277,"threads_started":1}
W20260812 02:37:22.930745 19005 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:37:22.921001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:37243 (local address 127.7.132.190:37037)
0812 02:37:22.921253 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:22.921259 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:22.921540 (+ 281us) server_negotiation.cc:407] Connection header received
0812 02:37:22.921679 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:22.921683 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:22.921764 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:22.921849 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:37:22.922741 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:22.923580 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:22.924878 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:22.925318 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:22.925431 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:22.930408 (+ 4977us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.132.130:37243: BlockingRecv error: recv got EOF from 127.7.132.130:37243 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
Aug 12 02:37:23 dist-test-slave-mn4r krb5kdc[17758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.129@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 02:37:23 dist-test-slave-mn4r krb5kdc[17758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.129@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 02:37:23.587332 19010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:23.578187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:37037 (local address 127.7.132.129:43449)
0812 02:37:23.578866 (+ 679us) negotiation.cc:107] Waiting for socket to connect
0812 02:37:23.578895 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0812 02:37:23.579030 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:37:23.579340 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:37:23.579343 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:37:23.579787 (+ 444us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:37:23.579956 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:23.579993 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:23.581057 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:23.581061 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:23.582178 (+ 1117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:23.582190 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:23.582888 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:23.582892 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:23.583028 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:23.583578 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:37:23.583595 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:37:23.587076 (+ 3481us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.132.190:37037: Server oryx/127.7.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":541,"thread_start_us":284,"threads_started":1}
W20260812 02:37:23.587860 19008 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:37:23.578232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:43449 (local address 127.7.132.190:37037)
0812 02:37:23.578997 (+ 765us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:23.579002 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:23.579019 (+ 17us) server_negotiation.cc:407] Connection header received
0812 02:37:23.579138 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:23.579142 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:23.579208 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:23.579295 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:37:23.580160 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:23.580936 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:23.582317 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:23.582754 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:23.582858 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:23.587340 (+ 4482us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.132.129:43449: BlockingRecv error: recv got EOF from 127.7.132.129:43449 (error 108)
Metrics: {"server-negotiator.queue_time_us":671,"thread_start_us":553,"threads_started":1}
Aug 12 02:37:23 dist-test-slave-mn4r krb5kdc[17758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.131@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 02:37:23 dist-test-slave-mn4r krb5kdc[17758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.131@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 02:37:23.799767 19011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:23.789863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:37037 (local address 127.7.132.131:48925)
0812 02:37:23.790459 (+ 596us) negotiation.cc:107] Waiting for socket to connect
0812 02:37:23.790475 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 02:37:23.790653 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:37:23.791054 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:37:23.791059 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:37:23.791633 (+ 574us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:37:23.791810 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:23.791819 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:23.793123 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:23.793129 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:23.794273 (+ 1144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:23.794287 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:23.795029 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:23.795036 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:23.795263 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:23.795852 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:37:23.795869 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:37:23.799529 (+ 3660us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.132.190:37037: Server oryx/127.7.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":331,"threads_started":1}
W20260812 02:37:23.799947 19008 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:37:23.790031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:48925 (local address 127.7.132.190:37037)
0812 02:37:23.790279 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:23.790286 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:23.790575 (+ 289us) server_negotiation.cc:407] Connection header received
0812 02:37:23.790808 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:23.790812 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:23.790876 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:23.790985 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:37:23.791973 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:23.792976 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:23.794428 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:23.794850 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:23.794989 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:23.799731 (+ 4742us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.132.131:48925: BlockingRecv error: recv got EOF from 127.7.132.131:48925 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Aug 12 02:37:23 dist-test-slave-mn4r krb5kdc[17758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.130@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 02:37:23 dist-test-slave-mn4r krb5kdc[17758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.130@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 02:37:23.940948 19012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:23.931427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:37037 (local address 127.7.132.130:41277)
0812 02:37:23.932046 (+ 619us) negotiation.cc:107] Waiting for socket to connect
0812 02:37:23.932063 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 02:37:23.932186 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:37:23.932521 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:37:23.932525 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:37:23.932981 (+ 456us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:37:23.933155 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:23.933164 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:23.934207 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:23.934210 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:23.935150 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:23.935162 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:23.935850 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:23.935854 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:23.935971 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:23.936601 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:37:23.936619 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:37:23.940594 (+ 3975us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.132.190:37037: Server oryx/127.7.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":513,"thread_start_us":307,"threads_started":1}
W20260812 02:37:23.941088 19008 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:37:23.931552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:41277 (local address 127.7.132.190:37037)
0812 02:37:23.931836 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:23.931843 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:23.932157 (+ 314us) server_negotiation.cc:407] Connection header received
0812 02:37:23.932296 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:23.932299 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:23.932375 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:23.932459 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:37:23.933286 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:23.934059 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:23.935296 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:23.935709 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:23.935804 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:23.940815 (+ 5011us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.132.130:41277: BlockingRecv error: recv got EOF from 127.7.132.130:41277 (error 108)
Metrics: {"server-negotiator.queue_time_us":110}
Aug 12 02:37:24 dist-test-slave-mn4r krb5kdc[17758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.129@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 02:37:24 dist-test-slave-mn4r krb5kdc[17758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.129@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 02:37:24.597592 19013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:24.588308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:37037 (local address 127.7.132.129:43551)
0812 02:37:24.588896 (+ 588us) negotiation.cc:107] Waiting for socket to connect
0812 02:37:24.588918 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 02:37:24.589040 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:37:24.589402 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:37:24.589405 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:37:24.589811 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:37:24.589981 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:24.589993 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:24.591398 (+ 1405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:24.591403 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:24.592316 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:24.592327 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:24.593028 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:24.593032 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:24.593142 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:24.593799 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:37:24.593816 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:37:24.597339 (+ 3523us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.132.190:37037: Server oryx/127.7.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":296,"threads_started":1}
W20260812 02:37:24.597831 19014 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:37:24.588387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:43551 (local address 127.7.132.190:37037)
0812 02:37:24.588922 (+ 535us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:24.588926 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:24.588993 (+ 67us) server_negotiation.cc:407] Connection header received
0812 02:37:24.589178 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:24.589183 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:24.589262 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:24.589355 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:37:24.590138 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:24.591260 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:24.592440 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:24.592871 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:24.592969 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:24.597555 (+ 4586us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.132.129:43551: BlockingRecv error: recv got EOF from 127.7.132.129:43551 (error 108)
Metrics: {"server-negotiator.queue_time_us":432,"thread_start_us":311,"threads_started":1}
Aug 12 02:37:24 dist-test-slave-mn4r krb5kdc[17758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.131@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 02:37:24 dist-test-slave-mn4r krb5kdc[17758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.131@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 02:37:24.810231 19015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:24.800715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:37037 (local address 127.7.132.131:47471)
0812 02:37:24.801360 (+ 645us) negotiation.cc:107] Waiting for socket to connect
0812 02:37:24.801376 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 02:37:24.801541 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:37:24.801904 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:37:24.801908 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:37:24.802439 (+ 531us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:37:24.802636 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:24.802646 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:24.803677 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:24.803682 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:24.804648 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:24.804660 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:24.805579 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:24.805586 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:24.805773 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:24.806495 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:37:24.806514 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:37:24.809921 (+ 3407us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.132.190:37037: Server oryx/127.7.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":554,"thread_start_us":299,"threads_started":1}
W20260812 02:37:24.810415 19014 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:37:24.800900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:47471 (local address 127.7.132.190:37037)
0812 02:37:24.801133 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:24.801142 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:24.801461 (+ 319us) server_negotiation.cc:407] Connection header received
0812 02:37:24.801675 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:24.801687 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:24.801749 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:24.801854 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:37:24.802788 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:24.803532 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:24.804799 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:24.805264 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:24.805560 (+ 296us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:24.810170 (+ 4610us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.132.131:47471: BlockingRecv error: recv got EOF from 127.7.132.131:47471 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 12 02:37:24 dist-test-slave-mn4r krb5kdc[17758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.130@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 02:37:24 dist-test-slave-mn4r krb5kdc[17758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.130@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 02:37:24.952234 19017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:24.942134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:37037 (local address 127.7.132.130:34159)
0812 02:37:24.942689 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0812 02:37:24.942728 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 02:37:24.942896 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:37:24.943304 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:37:24.943308 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:37:24.943703 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:37:24.943873 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:24.943881 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:24.945057 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:24.945061 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:24.946016 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:24.946028 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:24.946847 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:24.946852 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:24.946980 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:24.947715 (+ 735us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:37:24.947747 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:37:24.951810 (+ 4063us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.132.190:37037: Server oryx/127.7.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":293,"threads_started":1}
W20260812 02:37:24.952353 19014 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:37:24.942217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:34159 (local address 127.7.132.190:37037)
0812 02:37:24.942431 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:24.942439 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:24.942846 (+ 407us) server_negotiation.cc:407] Connection header received
0812 02:37:24.943051 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:24.943057 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:24.943153 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:24.943259 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:37:24.944009 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:24.944924 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:24.946240 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:24.946669 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:24.946764 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:24.952078 (+ 5314us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.132.130:34159: BlockingRecv error: recv got EOF from 127.7.132.130:34159 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 12 02:37:25 dist-test-slave-mn4r krb5kdc[17758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.129@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 02:37:25 dist-test-slave-mn4r krb5kdc[17758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.129@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 02:37:25.613078 19018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:25.598535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:37037 (local address 127.7.132.129:55429)
0812 02:37:25.599143 (+ 608us) negotiation.cc:107] Waiting for socket to connect
0812 02:37:25.599170 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0812 02:37:25.599305 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:37:25.599666 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:37:25.599671 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:37:25.600122 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:37:25.600272 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:25.600282 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:25.601500 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:25.601513 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:25.602629 (+ 1116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:25.602641 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:25.603451 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:25.603457 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:25.603670 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:25.608801 (+ 5131us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:37:25.608826 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:37:25.612801 (+ 3975us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.132.190:37037: Server oryx/127.7.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":279,"threads_started":1}
W20260812 02:37:25.613313 19019 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:37:25.598712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:55429 (local address 127.7.132.190:37037)
0812 02:37:25.599203 (+ 491us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:25.599211 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:25.599230 (+ 19us) server_negotiation.cc:407] Connection header received
0812 02:37:25.599436 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:25.599442 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:25.599519 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:25.599630 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:37:25.600421 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:25.601348 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:25.602785 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:25.603290 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:25.603434 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:25.613033 (+ 9599us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.132.129:55429: BlockingRecv error: recv got EOF from 127.7.132.129:55429 (error 108)
Metrics: {"server-negotiator.queue_time_us":375,"thread_start_us":262,"threads_started":1}
Aug 12 02:37:25 dist-test-slave-mn4r krb5kdc[17758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.131@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 02:37:25 dist-test-slave-mn4r krb5kdc[17758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.131@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 02:37:25.820639 19021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:25.811298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:37037 (local address 127.7.132.131:37347)
0812 02:37:25.811892 (+ 594us) negotiation.cc:107] Waiting for socket to connect
0812 02:37:25.811911 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 02:37:25.812031 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:37:25.812425 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:37:25.812432 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:37:25.812964 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:37:25.813189 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:25.813202 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:25.814312 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:25.814316 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:25.815246 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:25.815258 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:25.815981 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:25.815986 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:25.816109 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:25.816810 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:37:25.816827 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:37:25.820366 (+ 3539us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.132.190:37037: Server oryx/127.7.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":300,"threads_started":1}
W20260812 02:37:25.820845 19019 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:37:25.811345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:37347 (local address 127.7.132.190:37037)
0812 02:37:25.811588 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:25.811595 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:25.811982 (+ 387us) server_negotiation.cc:407] Connection header received
0812 02:37:25.812157 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:25.812162 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:25.812239 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:25.812320 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:37:25.813348 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:25.814188 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:25.815364 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:25.815759 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:25.815867 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:25.820585 (+ 4718us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.132.131:37347: BlockingRecv error: recv got EOF from 127.7.132.131:37347 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 12 02:37:25 dist-test-slave-mn4r krb5kdc[17758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.130@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 02:37:25 dist-test-slave-mn4r krb5kdc[17758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.130@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 02:37:25.963044 19022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:25.953334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:37037 (local address 127.7.132.130:37131)
0812 02:37:25.953940 (+ 606us) negotiation.cc:107] Waiting for socket to connect
0812 02:37:25.953956 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 02:37:25.954171 (+ 215us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:37:25.954497 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:37:25.954501 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:37:25.954901 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:37:25.955077 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:25.955085 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:25.956093 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:25.956097 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:25.957447 (+ 1350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:25.957462 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:25.958244 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:25.958248 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:25.958366 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:25.958904 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:37:25.958921 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:37:25.962804 (+ 3883us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.132.190:37037: Server oryx/127.7.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":345,"threads_started":1}
W20260812 02:37:25.963343 19019 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:37:25.953403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:37131 (local address 127.7.132.190:37037)
0812 02:37:25.953630 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:25.953639 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:25.954062 (+ 423us) server_negotiation.cc:407] Connection header received
0812 02:37:25.954291 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:25.954295 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:25.954372 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:25.954456 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:37:25.955193 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:25.955921 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:25.957579 (+ 1658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:25.958043 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:25.958204 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:25.963020 (+ 4816us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.132.130:37131: BlockingRecv error: recv got EOF from 127.7.132.130:37131 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 12 02:37:26 dist-test-slave-mn4r krb5kdc[17758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.129@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 02:37:26 dist-test-slave-mn4r krb5kdc[17758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.129@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 02:37:26.623560 19023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:26.614145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:37037 (local address 127.7.132.129:60615)
0812 02:37:26.614644 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0812 02:37:26.614671 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0812 02:37:26.614776 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:37:26.615251 (+ 475us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:37:26.615254 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:37:26.615598 (+ 344us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:37:26.615761 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:26.615769 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:26.616836 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:26.616839 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:26.617833 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:26.617845 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:26.618572 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:26.618576 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:26.618694 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:26.619319 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:37:26.619338 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:37:26.623288 (+ 3950us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.132.190:37037: Server oryx/127.7.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":269,"threads_started":1}
W20260812 02:37:26.623780 19024 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:37:26.614194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:60615 (local address 127.7.132.190:37037)
0812 02:37:26.614973 (+ 779us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:26.614977 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:26.614998 (+ 21us) server_negotiation.cc:407] Connection header received
0812 02:37:26.615044 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:26.615047 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:26.615113 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:26.615201 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:37:26.615901 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:26.616721 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:26.617962 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:26.618436 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:26.618528 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:26.623532 (+ 5004us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.132.129:60615: BlockingRecv error: recv got EOF from 127.7.132.129:60615 (error 108)
Metrics: {"server-negotiator.queue_time_us":691,"thread_start_us":569,"threads_started":1}
Aug 12 02:37:26 dist-test-slave-mn4r krb5kdc[17758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.131@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 02:37:26 dist-test-slave-mn4r krb5kdc[17758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.131@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 02:37:26.831528 19025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:26.821721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:37037 (local address 127.7.132.131:41095)
0812 02:37:26.822318 (+ 597us) negotiation.cc:107] Waiting for socket to connect
0812 02:37:26.822335 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 02:37:26.822449 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:37:26.822769 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:37:26.822772 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:37:26.823212 (+ 440us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:37:26.823364 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:26.823373 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:26.824462 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:26.824467 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:26.825571 (+ 1104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:26.825593 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:26.826648 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:26.826656 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:26.826866 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:26.827592 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:37:26.827610 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:37:26.831284 (+ 3674us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.132.190:37037: Server oryx/127.7.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":350,"threads_started":1}
W20260812 02:37:26.831724 19024 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:37:26.821756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:41095 (local address 127.7.132.190:37037)
0812 02:37:26.821949 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:26.821954 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:26.822408 (+ 454us) server_negotiation.cc:407] Connection header received
0812 02:37:26.822559 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:26.822563 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:26.822624 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:26.822709 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:37:26.823491 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:26.824296 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:26.825757 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:26.826450 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:26.826585 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:26.831484 (+ 4899us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.132.131:41095: BlockingRecv error: recv got EOF from 127.7.132.131:41095 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 12 02:37:26 dist-test-slave-mn4r krb5kdc[17758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.130@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 02:37:26 dist-test-slave-mn4r krb5kdc[17758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.130@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 02:37:26.973516 19027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:26.963911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:37037 (local address 127.7.132.130:59701)
0812 02:37:26.964530 (+ 619us) negotiation.cc:107] Waiting for socket to connect
0812 02:37:26.964547 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 02:37:26.964665 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:37:26.965073 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:37:26.965076 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:37:26.965437 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:37:26.965617 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:26.965627 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:26.966813 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:26.966818 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:26.967839 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:26.967853 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:26.968540 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:26.968544 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:26.968661 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:26.969378 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:37:26.969396 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:37:26.973263 (+ 3867us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.132.190:37037: Server oryx/127.7.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":351,"threads_started":1}
W20260812 02:37:26.973614 19024 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:37:26.963980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:59701 (local address 127.7.132.190:37037)
0812 02:37:26.964151 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:26.964156 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:26.964623 (+ 467us) server_negotiation.cc:407] Connection header received
0812 02:37:26.964816 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:26.964821 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:26.964938 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:26.965031 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:37:26.965755 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:26.966651 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:26.967988 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:26.968386 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:26.968496 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:26.973408 (+ 4912us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.132.130:59701: BlockingRecv error: recv got EOF from 127.7.132.130:59701 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 12 02:37:27 dist-test-slave-mn4r krb5kdc[17758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.129@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 02:37:27 dist-test-slave-mn4r krb5kdc[17758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.129@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 02:37:27.633886 19028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:27.624489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:37037 (local address 127.7.132.129:52985)
0812 02:37:27.625025 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0812 02:37:27.625046 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 02:37:27.625149 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:37:27.625722 (+ 573us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:37:27.625725 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:37:27.626212 (+ 487us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:37:27.626399 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:27.626409 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:27.627523 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:27.627527 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:27.628452 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:27.628464 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:27.629200 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:27.629205 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:27.629325 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:27.629981 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:37:27.629999 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:37:27.633549 (+ 3550us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.132.190:37037: Server oryx/127.7.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":259,"threads_started":1}
W20260812 02:37:27.634236 19029 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:37:27.624697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:52985 (local address 127.7.132.190:37037)
0812 02:37:27.625420 (+ 723us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:27.625426 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:27.625441 (+ 15us) server_negotiation.cc:407] Connection header received
0812 02:37:27.625486 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:27.625490 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:27.625553 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:27.625652 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:37:27.626582 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:27.627382 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:27.628591 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:27.629050 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:27.629152 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:27.633875 (+ 4723us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.132.129:52985: BlockingRecv error: recv got EOF from 127.7.132.129:52985 (error 108)
Metrics: {"server-negotiator.queue_time_us":631,"thread_start_us":489,"threads_started":1}
Aug 12 02:37:27 dist-test-slave-mn4r krb5kdc[17758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.131@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 02:37:27 dist-test-slave-mn4r krb5kdc[17758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.131@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 02:37:27.842324 19030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:27.832544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:37037 (local address 127.7.132.131:51001)
0812 02:37:27.833179 (+ 635us) negotiation.cc:107] Waiting for socket to connect
0812 02:37:27.833196 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 02:37:27.833370 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:37:27.833737 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:37:27.833742 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:37:27.834206 (+ 464us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:37:27.834375 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:27.834385 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:27.835463 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:27.835475 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:27.836495 (+ 1020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:27.836507 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:27.837264 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:27.837270 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:27.837450 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:27.838052 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:37:27.838069 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:37:27.841945 (+ 3876us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.132.190:37037: Server oryx/127.7.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":533,"thread_start_us":420,"threads_started":1}
W20260812 02:37:27.842409 19029 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:37:27.832680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:51001 (local address 127.7.132.190:37037)
0812 02:37:27.832931 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:27.832937 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:27.833298 (+ 361us) server_negotiation.cc:407] Connection header received
0812 02:37:27.833515 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:27.833521 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:27.833593 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:27.833699 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:37:27.834532 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:27.835305 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:27.836648 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:27.837104 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:27.837245 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:27.842162 (+ 4917us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.132.131:51001: BlockingRecv error: recv got EOF from 127.7.132.131:51001 (error 108)
Metrics: {"server-negotiator.queue_time_us":91}
Aug 12 02:37:27 dist-test-slave-mn4r krb5kdc[17758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.130@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 02:37:27 dist-test-slave-mn4r krb5kdc[17758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.130@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 02:37:27.983515 19033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:27.974500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:37037 (local address 127.7.132.130:60811)
0812 02:37:27.974968 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0812 02:37:27.974986 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 02:37:27.975109 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:37:27.975390 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:37:27.975393 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:37:27.975803 (+ 410us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:37:27.975980 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:27.975992 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:27.977003 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:27.977007 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:27.977943 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:27.977956 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:27.978650 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:27.978654 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:27.978783 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:27.979452 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:37:27.979477 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:37:27.983163 (+ 3686us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.132.190:37037: Server oryx/127.7.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":261,"threads_started":1}
W20260812 02:37:27.983661 19029 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:37:27.974648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:60811 (local address 127.7.132.190:37037)
0812 02:37:27.974878 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:27.974884 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:27.975127 (+ 243us) server_negotiation.cc:407] Connection header received
0812 02:37:27.975209 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:27.975213 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:27.975292 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:27.975377 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:37:27.976124 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:27.976874 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:27.978113 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:27.978510 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:27.978611 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:27.983392 (+ 4781us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.132.130:60811: BlockingRecv error: recv got EOF from 127.7.132.130:60811 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
Aug 12 02:37:28 dist-test-slave-mn4r krb5kdc[17758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.129@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 02:37:28 dist-test-slave-mn4r krb5kdc[17758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.129@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 02:37:28.644436 19034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:28.635021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:37037 (local address 127.7.132.129:59825)
0812 02:37:28.635569 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0812 02:37:28.635591 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 02:37:28.635708 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:37:28.636025 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:37:28.636028 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:37:28.636438 (+ 410us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:37:28.636628 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:28.636637 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:28.637646 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:28.637650 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:28.638679 (+ 1029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:28.638691 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:28.639375 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:28.639380 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:28.639556 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:28.640329 (+ 773us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:37:28.640347 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:37:28.644179 (+ 3832us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.132.190:37037: Server oryx/127.7.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":301,"threads_started":1}
W20260812 02:37:28.644657 19035 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:37:28.635077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:59825 (local address 127.7.132.190:37037)
0812 02:37:28.635598 (+ 521us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:28.635602 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:28.635646 (+ 44us) server_negotiation.cc:407] Connection header received
0812 02:37:28.635819 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:28.635823 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:28.635883 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:28.636003 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:37:28.636757 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:28.637523 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:28.638800 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:28.639254 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:28.639342 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:28.644400 (+ 5058us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.132.129:59825: BlockingRecv error: recv got EOF from 127.7.132.129:59825 (error 108)
Metrics: {"server-negotiator.queue_time_us":431,"thread_start_us":294,"threads_started":1}
Aug 12 02:37:28 dist-test-slave-mn4r krb5kdc[17758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.131@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 02:37:28 dist-test-slave-mn4r krb5kdc[17758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.131@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 02:37:28.852388 19036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:28.843277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:37037 (local address 127.7.132.131:52269)
0812 02:37:28.843781 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0812 02:37:28.843798 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 02:37:28.843917 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:37:28.844236 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:37:28.844240 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:37:28.844684 (+ 444us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:37:28.844870 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:28.844879 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:28.845943 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:28.845947 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:28.847043 (+ 1096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:28.847056 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:28.847946 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:28.847951 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:28.848083 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:28.848673 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:37:28.848691 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:37:28.852041 (+ 3350us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.132.190:37037: Server oryx/127.7.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":408,"thread_start_us":317,"threads_started":1}
W20260812 02:37:28.852450 19035 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:37:28.843332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:52269 (local address 127.7.132.190:37037)
0812 02:37:28.843549 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:28.843557 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:28.843878 (+ 321us) server_negotiation.cc:407] Connection header received
0812 02:37:28.844040 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:28.844043 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:28.844106 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:28.844186 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:37:28.845008 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:28.845791 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:28.847185 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:28.847781 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:28.847881 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:28.852221 (+ 4340us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.132.131:52269: BlockingRecv error: recv got EOF from 127.7.132.131:52269 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 12 02:37:28 dist-test-slave-mn4r krb5kdc[17758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.130@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 02:37:28 dist-test-slave-mn4r krb5kdc[17758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.130@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 02:37:28.993840 19037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:28.984610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:37037 (local address 127.7.132.130:37887)
0812 02:37:28.985190 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0812 02:37:28.985207 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 02:37:28.985327 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:37:28.985663 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:37:28.985667 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:37:28.986107 (+ 440us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:37:28.986286 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:28.986295 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:28.987336 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:28.987340 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:28.988301 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:28.988314 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:28.989028 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:28.989033 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:28.989165 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:28.989813 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:37:28.989832 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:37:28.993585 (+ 3753us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.132.190:37037: Server oryx/127.7.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":325,"threads_started":1}
W20260812 02:37:28.994046 19035 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:37:28.984635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:37887 (local address 127.7.132.190:37037)
0812 02:37:28.984946 (+ 311us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:28.984960 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:28.985307 (+ 347us) server_negotiation.cc:407] Connection header received
0812 02:37:28.985437 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:28.985441 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:28.985518 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:28.985601 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:37:28.986419 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:28.987207 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:28.988437 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:28.988859 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:28.988961 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:28.993788 (+ 4827us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.132.130:37887: BlockingRecv error: recv got EOF from 127.7.132.130:37887 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Aug 12 02:37:29 dist-test-slave-mn4r krb5kdc[17758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.129@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 02:37:29 dist-test-slave-mn4r krb5kdc[17758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.129@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 02:37:29.654749 19038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:29.645488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:37037 (local address 127.7.132.129:38741)
0812 02:37:29.646037 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0812 02:37:29.646068 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0812 02:37:29.646237 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:37:29.646708 (+ 471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:37:29.646713 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:37:29.647148 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:37:29.647293 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:29.647303 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:29.648373 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:29.648378 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:29.649301 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:29.649312 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:29.650052 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:29.650061 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:29.650276 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:29.650852 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:37:29.650869 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:37:29.654414 (+ 3545us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.132.190:37037: Server oryx/127.7.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":277,"threads_started":1}
W20260812 02:37:29.655136 19039 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:37:29.645539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:38741 (local address 127.7.132.190:37037)
0812 02:37:29.646414 (+ 875us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:29.646419 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:29.646434 (+ 15us) server_negotiation.cc:407] Connection header received
0812 02:37:29.646487 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:29.646491 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:29.646546 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:29.646670 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:37:29.647461 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:29.648226 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:29.649448 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:29.649883 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:29.650019 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:29.654652 (+ 4633us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.132.129:38741: BlockingRecv error: recv got EOF from 127.7.132.129:38741 (error 108)
Metrics: {"server-negotiator.queue_time_us":787,"thread_start_us":649,"threads_started":1}
Aug 12 02:37:29 dist-test-slave-mn4r krb5kdc[17758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.131@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 02:37:29 dist-test-slave-mn4r krb5kdc[17758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.131@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 02:37:29.862382 19043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:29.853344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:37037 (local address 127.7.132.131:50695)
0812 02:37:29.853910 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0812 02:37:29.853926 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 02:37:29.854045 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:37:29.854389 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:37:29.854396 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:37:29.854906 (+ 510us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:37:29.855091 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:29.855102 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:29.856111 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:29.856115 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:29.857051 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:29.857073 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:29.857775 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:29.857779 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:29.857915 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:29.858583 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:37:29.858602 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:37:29.862126 (+ 3524us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.132.190:37037: Server oryx/127.7.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":303,"threads_started":1}
W20260812 02:37:29.862526 19039 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:37:29.853440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:50695 (local address 127.7.132.190:37037)
0812 02:37:29.853633 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:29.853637 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:29.854020 (+ 383us) server_negotiation.cc:407] Connection header received
0812 02:37:29.854172 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:29.854176 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:29.854237 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:29.854316 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:37:29.855220 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:29.855978 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:29.857199 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:29.857616 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:29.857709 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:29.862304 (+ 4595us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.132.131:50695: BlockingRecv error: recv got EOF from 127.7.132.131:50695 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Aug 12 02:37:30 dist-test-slave-mn4r krb5kdc[17758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.130@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 02:37:30 dist-test-slave-mn4r krb5kdc[17758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.130@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 02:37:30.004963 19044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:29.994915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:37037 (local address 127.7.132.130:55493)
0812 02:37:29.995529 (+ 614us) negotiation.cc:107] Waiting for socket to connect
0812 02:37:29.995550 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 02:37:29.995739 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:37:29.996181 (+ 442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:37:29.996186 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:37:29.996692 (+ 506us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:37:29.996839 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:29.996848 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:29.998142 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:29.998151 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:29.999072 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:29.999083 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:29.999794 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:29.999800 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:29.999998 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:30.000647 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:37:30.000665 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:37:30.004695 (+ 4030us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.132.190:37037: Server oryx/127.7.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":499,"thread_start_us":300,"threads_started":1}
W20260812 02:37:30.005211 19039 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:37:29.995307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:55493 (local address 127.7.132.190:37037)
0812 02:37:29.995538 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:29.995546 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:29.995656 (+ 110us) server_negotiation.cc:407] Connection header received
0812 02:37:29.995904 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:29.995908 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:29.996037 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:29.996146 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:37:29.997011 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:29.997956 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:29.999223 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:29.999643 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:29.999778 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:30.004931 (+ 5153us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.132.130:55493: BlockingRecv error: recv got EOF from 127.7.132.130:55493 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Aug 12 02:37:30 dist-test-slave-mn4r krb5kdc[17758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.129@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 02:37:30 dist-test-slave-mn4r krb5kdc[17758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.129@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 02:37:30.670639 19046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:30.656349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:37037 (local address 127.7.132.129:60875)
0812 02:37:30.656904 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0812 02:37:30.656928 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0812 02:37:30.657032 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:37:30.657723 (+ 691us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:37:30.657726 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:37:30.658121 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:37:30.658297 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:30.658306 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:30.659473 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:30.659477 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:30.660471 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:30.660482 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:30.661306 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:30.661310 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:30.661462 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:30.665786 (+ 4324us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:37:30.665807 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:37:30.670227 (+ 4420us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.132.190:37037: Server oryx/127.7.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":300,"threads_started":1}
W20260812 02:37:30.670773 19047 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:37:30.656797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:60875 (local address 127.7.132.190:37037)
0812 02:37:30.657427 (+ 630us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:30.657432 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:30.657448 (+ 16us) server_negotiation.cc:407] Connection header received
0812 02:37:30.657499 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:30.657503 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:30.657565 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:30.657662 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:37:30.658452 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:30.659318 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:30.660610 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:30.661171 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:30.661284 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:30.670521 (+ 9237us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.132.129:60875: BlockingRecv error: recv got EOF from 127.7.132.129:60875 (error 108)
Metrics: {"server-negotiator.queue_time_us":531,"thread_start_us":366,"threads_started":1}
Aug 12 02:37:30 dist-test-slave-mn4r krb5kdc[17758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.131@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 02:37:30 dist-test-slave-mn4r krb5kdc[17758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.131@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 02:37:30.872483 19048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:30.863497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:37037 (local address 127.7.132.131:54721)
0812 02:37:30.864226 (+ 729us) negotiation.cc:107] Waiting for socket to connect
0812 02:37:30.864252 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0812 02:37:30.864371 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:37:30.864700 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:37:30.864703 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:37:30.865147 (+ 444us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:37:30.865301 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:30.865309 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:30.866325 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:30.866329 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:30.867261 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:30.867272 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:30.867971 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:30.867975 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:30.868096 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:30.868746 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:37:30.868765 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:37:30.872202 (+ 3437us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.132.190:37037: Server oryx/127.7.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":622,"thread_start_us":516,"threads_started":1}
W20260812 02:37:30.872774 19047 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:37:30.863553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:54721 (local address 127.7.132.190:37037)
0812 02:37:30.863907 (+ 354us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:30.863917 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:30.864342 (+ 425us) server_negotiation.cc:407] Connection header received
0812 02:37:30.864481 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:30.864485 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:30.864561 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:30.864644 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:37:30.865428 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:30.866196 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:30.867395 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:30.867825 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:30.867918 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:30.872498 (+ 4580us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.132.131:54721: BlockingRecv error: recv got EOF from 127.7.132.131:54721 (error 108)
Metrics: {"server-negotiator.queue_time_us":173}
Aug 12 02:37:31 dist-test-slave-mn4r krb5kdc[17758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.130@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 02:37:31 dist-test-slave-mn4r krb5kdc[17758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.130@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 02:37:31.015799 19050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:31.006354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:37037 (local address 127.7.132.130:53357)
0812 02:37:31.006906 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0812 02:37:31.006923 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 02:37:31.007076 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:37:31.007517 (+ 441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:37:31.007524 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:37:31.008103 (+ 579us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:37:31.008339 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:31.008353 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:31.009641 (+ 1288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:31.009645 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:31.010598 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:31.010610 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:31.011249 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:31.011253 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:31.011372 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:31.012038 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:37:31.012056 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:37:31.015551 (+ 3495us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.132.190:37037: Server oryx/127.7.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":292,"threads_started":1}
W20260812 02:37:31.016104 19047 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:37:31.006452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:53357 (local address 127.7.132.190:37037)
0812 02:37:31.006648 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:31.006653 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:31.007046 (+ 393us) server_negotiation.cc:407] Connection header received
0812 02:37:31.007215 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:31.007221 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:31.007320 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:31.007428 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:37:31.008547 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:31.009507 (+ 960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:31.010728 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:31.011125 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:31.011219 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:31.015748 (+ 4529us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.132.130:53357: BlockingRecv error: recv got EOF from 127.7.132.130:53357 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
I20260812 02:37:31.546761 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 18470
I20260812 02:37:31.559818 19053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:31.548159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33978 (local address 127.7.132.129:39069)
0812 02:37:31.548802 (+ 643us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:31.548810 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:31.548838 (+ 28us) server_negotiation.cc:407] Connection header received
0812 02:37:31.548913 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:31.548920 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:31.549171 (+ 251us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:31.549347 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:37:31.550996 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:31.552051 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:31.553305 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:31.553858 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:31.553965 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:31.556502 (+ 2537us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:37:31.556540 (+ 38us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:37:31.556560 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:37:31.556628 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:37:31.559272 (+ 2644us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.7.132.129@KRBTEST.COM not found in keytab (ticket kvno 2)
0812 02:37:31.559571 (+ 299us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:33978: Request ticket server kudu/127.7.132.129@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":467,"thread_start_us":340,"threads_started":1}
W20260812 02:37:31.559981 7698 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 18470: Not authorized: Client connection negotiation failed: client connection to 127.7.132.129:39069: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.7.132.129@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260812 02:37:31.560051 19053 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:33978: Request ticket server kudu/127.7.132.129@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260812 02:37:31.560204 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 18470
I20260812 02:37:31.582100 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 18657
I20260812 02:37:31.594668 19054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:31.583176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51938 (local address 127.7.132.130:42401)
0812 02:37:31.583766 (+ 590us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:31.583773 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:31.583805 (+ 32us) server_negotiation.cc:407] Connection header received
0812 02:37:31.583868 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:31.583874 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:31.584133 (+ 259us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:31.584357 (+ 224us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:37:31.585684 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:31.586766 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:31.588001 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:31.588563 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:31.588708 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:31.591269 (+ 2561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:37:31.591278 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:37:31.591292 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:37:31.591333 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:37:31.593823 (+ 2490us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.7.132.130@KRBTEST.COM not found in keytab (ticket kvno 2)
0812 02:37:31.594341 (+ 518us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:51938: Request ticket server kudu/127.7.132.130@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":434,"thread_start_us":325,"threads_started":1}
W20260812 02:37:31.594781 7698 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 18657: Not authorized: Client connection negotiation failed: client connection to 127.7.132.130:42401: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.7.132.130@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260812 02:37:31.594807 19054 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:51938: Request ticket server kudu/127.7.132.130@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260812 02:37:31.594972 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 18657
I20260812 02:37:31.617156 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 18814
I20260812 02:37:31.630903 19056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:31.618821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37930 (local address 127.7.132.131:44117)
0812 02:37:31.619458 (+ 637us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:31.619471 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:31.619496 (+ 25us) server_negotiation.cc:407] Connection header received
0812 02:37:31.619560 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:31.619567 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:31.619818 (+ 251us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:31.619986 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:37:31.621874 (+ 1888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:31.623023 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:31.624267 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:31.624838 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:31.624940 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:31.627638 (+ 2698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:37:31.627647 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:37:31.627661 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:37:31.627696 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:37:31.630322 (+ 2626us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.7.132.131@KRBTEST.COM not found in keytab (ticket kvno 2)
0812 02:37:31.630621 (+ 299us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:37930: Request ticket server kudu/127.7.132.131@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":435,"thread_start_us":320,"threads_started":1}
W20260812 02:37:31.630944 7698 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 18814: Not authorized: Client connection negotiation failed: client connection to 127.7.132.131:44117: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.7.132.131@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260812 02:37:31.631012 19056 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:37930: Request ticket server kudu/127.7.132.131@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260812 02:37:31.631179 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 18814
I20260812 02:37:31.652415 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 18390
I20260812 02:37:31.653893 18462 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:37:31.772655 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 18390
2026-08-12T02:37:31Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (22007 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-12T02:37:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T02:37:31Z Disabled control of system clock
I20260812 02:37:31.869624 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502156971023-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502156971023-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502156971023-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:38469
--webserver_interface=127.7.132.190
--webserver_port=0
--builtin_ntp_servers=127.7.132.148:33365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.132.190:38469
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260812 02:37:32.219763 19063 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:37:32.220064 19063 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:37:32.220212 19063 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:37:32.227694 19063 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:37:32.227797 19063 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:37:32.227882 19063 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:37:32.227998 19063 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:37:32.228062 19063 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:37:32.237375 19063 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:33365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502156971023-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502156971023-7698-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.132.190:38469
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:38469
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.132.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502156971023-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:37:32.238849 19063 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:37:32.240720 19063 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:37:32.253166 19069 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:37:32.253113 19073 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:37:32.253207 19071 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:37:32.253732 19063 server_base.cc:1034] running on GCE node
I20260812 02:37:32.254616 19063 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:37:32.256034 19063 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:37:32.257288 19063 hybrid_clock.cc:648] HybridClock initialized: now 1786502252257253 us; error 53 us; skew 500 ppm
I20260812 02:37:32.257684 19063 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 02:37:32.260116 19063 webserver.cc:460] Webserver started at http://127.7.132.190:34731/ using document root <none> and password file <none>
I20260812 02:37:32.260866 19063 fs_manager.cc:359] Metadata directory not provided
I20260812 02:37:32.261008 19063 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:37:32.261348 19063 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:37:32.264421 19063 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502156971023-7698-0/minicluster-data/master-0/data/instance:
uuid: "dd698f05e54d4b80b008978fbb84b0f9"
format_stamp: "Formatted at 2026-08-12 02:37:32 on dist-test-slave-mn4r"
server_key: "0c4816f9855dee609c8272c5e5c69911"
server_key_iv: "c62dd265416e5504e2375ee4a818c639"
server_key_version: "encryptionkey@0"
I20260812 02:37:32.265233 19063 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502156971023-7698-0/minicluster-data/master-0/wal/instance:
uuid: "dd698f05e54d4b80b008978fbb84b0f9"
format_stamp: "Formatted at 2026-08-12 02:37:32 on dist-test-slave-mn4r"
server_key: "0c4816f9855dee609c8272c5e5c69911"
server_key_iv: "c62dd265416e5504e2375ee4a818c639"
server_key_version: "encryptionkey@0"
I20260812 02:37:32.270569 19063 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260812 02:37:32.274068 19079 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:37:32.275441 19063 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 02:37:32.275650 19063 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502156971023-7698-0/minicluster-data/master-0/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502156971023-7698-0/minicluster-data/master-0/wal
uuid: "dd698f05e54d4b80b008978fbb84b0f9"
format_stamp: "Formatted at 2026-08-12 02:37:32 on dist-test-slave-mn4r"
server_key: "0c4816f9855dee609c8272c5e5c69911"
server_key_iv: "c62dd265416e5504e2375ee4a818c639"
server_key_version: "encryptionkey@0"
I20260812 02:37:32.275883 19063 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502156971023-7698-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502156971023-7698-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502156971023-7698-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:37:32.286749 19063 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:37:32.287559 19063 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:37:32.287863 19063 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:37:32.306663 19063 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.190:38469
I20260812 02:37:32.306681 19145 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.190:38469 every 8 connection(s)
I20260812 02:37:32.308046 19063 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
I20260812 02:37:32.308458 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 19063
I20260812 02:37:32.308648 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502156971023-7698-0/minicluster-data/master-0/wal/instance
I20260812 02:37:32.309599 7698 external_mini_cluster.cc:1428] Setting key 26623cd3af77c44ab6a858efcfecb33b
I20260812 02:37:32.313017 19146 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:37:32.323349 19149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:32.311881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55898 (local address 127.7.132.190:38469)
0812 02:37:32.312835 (+ 954us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:32.312846 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:32.312888 (+ 42us) server_negotiation.cc:407] Connection header received
0812 02:37:32.314642 (+ 1754us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:32.314664 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:32.315098 (+ 434us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:32.315519 (+ 421us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:37:32.316113 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:32.317575 (+ 1462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:32.318963 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:32.319610 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:32.319810 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:32.321051 (+ 1241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:37:32.321062 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:37:32.321077 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:37:32.321644 (+ 567us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:37:32.321720 (+ 76us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:37:32.321748 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:37:32.321908 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0812 02:37:32.322158 (+ 250us) server_negotiation.cc:299] Negotiation successful
0812 02:37:32.322445 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":540,"thread_start_us":337,"threads_started":1}
I20260812 02:37:32.323908 19146 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd698f05e54d4b80b008978fbb84b0f9: Bootstrap starting.
I20260812 02:37:32.327093 19146 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dd698f05e54d4b80b008978fbb84b0f9: Neither blocks nor log segments found. Creating new log.
I20260812 02:37:32.328090 19146 log.cc:824] T 00000000000000000000000000000000 P dd698f05e54d4b80b008978fbb84b0f9: Log is configured to *not* fsync() on all Append() calls
I20260812 02:37:32.330853 19146 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd698f05e54d4b80b008978fbb84b0f9: No bootstrap required, opened a new log
I20260812 02:37:32.338814 19146 raft_consensus.cc:348] T 00000000000000000000000000000000 P dd698f05e54d4b80b008978fbb84b0f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd698f05e54d4b80b008978fbb84b0f9" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 38469 } }
I20260812 02:37:32.339206 19146 raft_consensus.cc:374] T 00000000000000000000000000000000 P dd698f05e54d4b80b008978fbb84b0f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:37:32.339320 19146 raft_consensus.cc:729] T 00000000000000000000000000000000 P dd698f05e54d4b80b008978fbb84b0f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd698f05e54d4b80b008978fbb84b0f9, State: Initialized, Role: FOLLOWER
I20260812 02:37:32.339888 19146 consensus_queue.cc:260] T 00000000000000000000000000000000 P dd698f05e54d4b80b008978fbb84b0f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd698f05e54d4b80b008978fbb84b0f9" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 38469 } }
I20260812 02:37:32.340144 19146 raft_consensus.cc:388] T 00000000000000000000000000000000 P dd698f05e54d4b80b008978fbb84b0f9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 02:37:32.340297 19146 raft_consensus.cc:482] T 00000000000000000000000000000000 P dd698f05e54d4b80b008978fbb84b0f9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 02:37:32.340482 19146 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dd698f05e54d4b80b008978fbb84b0f9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:37:32.342778 19146 raft_consensus.cc:504] T 00000000000000000000000000000000 P dd698f05e54d4b80b008978fbb84b0f9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd698f05e54d4b80b008978fbb84b0f9" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 38469 } }
I20260812 02:37:32.343294 19146 leader_election.cc:302] T 00000000000000000000000000000000 P dd698f05e54d4b80b008978fbb84b0f9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dd698f05e54d4b80b008978fbb84b0f9; no voters:
I20260812 02:37:32.344002 19146 leader_election.cc:288] T 00000000000000000000000000000000 P dd698f05e54d4b80b008978fbb84b0f9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 02:37:32.344128 19151 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dd698f05e54d4b80b008978fbb84b0f9 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:37:32.345273 19151 raft_consensus.cc:686] T 00000000000000000000000000000000 P dd698f05e54d4b80b008978fbb84b0f9 [term 1 LEADER]: Becoming Leader. State: Replica: dd698f05e54d4b80b008978fbb84b0f9, State: Running, Role: LEADER
I20260812 02:37:32.345837 19151 consensus_queue.cc:237] T 00000000000000000000000000000000 P dd698f05e54d4b80b008978fbb84b0f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd698f05e54d4b80b008978fbb84b0f9" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 38469 } }
I20260812 02:37:32.346449 19146 sys_catalog.cc:564] T 00000000000000000000000000000000 P dd698f05e54d4b80b008978fbb84b0f9 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:37:32.349136 19153 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd698f05e54d4b80b008978fbb84b0f9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dd698f05e54d4b80b008978fbb84b0f9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd698f05e54d4b80b008978fbb84b0f9" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 38469 } } }
I20260812 02:37:32.349584 19153 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd698f05e54d4b80b008978fbb84b0f9 [sys.catalog]: This master's current role is: LEADER
I20260812 02:37:32.349314 19154 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd698f05e54d4b80b008978fbb84b0f9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dd698f05e54d4b80b008978fbb84b0f9. Latest consensus state: current_term: 1 leader_uuid: "dd698f05e54d4b80b008978fbb84b0f9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd698f05e54d4b80b008978fbb84b0f9" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 38469 } } }
I20260812 02:37:32.350728 19154 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd698f05e54d4b80b008978fbb84b0f9 [sys.catalog]: This master's current role is: LEADER
I20260812 02:37:32.350927 19161 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:37:32.355624 19161 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 02:37:32.365828 19161 catalog_manager.cc:1347] Generated new cluster ID: d9d51c758f3a4e05b9918b1598cd31fa
I20260812 02:37:32.366137 19161 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 02:37:32.398170 19161 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 02:37:32.399523 19161 catalog_manager.cc:1504] Loading token signing keys...
I20260812 02:37:32.405529 19161 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dd698f05e54d4b80b008978fbb84b0f9: Generated new TSK 0
I20260812 02:37:32.406623 19161 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 02:37:32.425076 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502156971023-7698-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502156971023-7698-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502156971023-7698-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502156971023-7698-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.129:0
--local_ip_for_outbound_sockets=127.7.132.129
--webserver_interface=127.7.132.129
--webserver_port=0
--tserver_master_addrs=127.7.132.190:38469
--builtin_ntp_servers=127.7.132.148:33365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260812 02:37:32.804852 19175 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:37:32.805142 19175 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:37:32.805227 19175 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:37:32.812557 19175 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:37:32.812659 19175 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:37:32.812765 19175 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.129
I20260812 02:37:32.821857 19175 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:33365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502156971023-7698-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502156971023-7698-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502156971023-7698-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.132.129
--webserver_port=0
--tserver_master_addrs=127.7.132.190:38469
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.129
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502156971023-7698-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:37:32.823190 19175 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:37:32.825078 19175 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:37:32.838783 19180 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:37:32.838886 19183 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:37:32.838907 19181 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:37:32.840000 19175 server_base.cc:1034] running on GCE node
I20260812 02:37:32.840677 19175 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:37:32.841665 19175 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:37:32.842980 19175 hybrid_clock.cc:648] HybridClock initialized: now 1786502252842876 us; error 92 us; skew 500 ppm
I20260812 02:37:32.843282 19175 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 02:37:32.845597 19175 webserver.cc:460] Webserver started at http://127.7.132.129:34229/ using document root <none> and password file <none>
I20260812 02:37:32.846361 19175 fs_manager.cc:359] Metadata directory not provided
I20260812 02:37:32.846503 19175 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:37:32.846910 19175 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:37:32.850073 19175 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502156971023-7698-0/minicluster-data/ts-0/data/instance:
uuid: "a57f4281d746461cbe72884b8311a6f2"
format_stamp: "Formatted at 2026-08-12 02:37:32 on dist-test-slave-mn4r"
server_key: "322dd2f9226f570d487955d6bb3bb105"
server_key_iv: "403b7eb5f09b4d0306b44170faa48e06"
server_key_version: "encryptionkey@0"
I20260812 02:37:32.851106 19175 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502156971023-7698-0/minicluster-data/ts-0/wal/instance:
uuid: "a57f4281d746461cbe72884b8311a6f2"
format_stamp: "Formatted at 2026-08-12 02:37:32 on dist-test-slave-mn4r"
server_key: "322dd2f9226f570d487955d6bb3bb105"
server_key_iv: "403b7eb5f09b4d0306b44170faa48e06"
server_key_version: "encryptionkey@0"
I20260812 02:37:32.856719 19175 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 02:37:32.860399 19191 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:37:32.861866 19175 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260812 02:37:32.862068 19175 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502156971023-7698-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502156971023-7698-0/minicluster-data/ts-0/wal
uuid: "a57f4281d746461cbe72884b8311a6f2"
format_stamp: "Formatted at 2026-08-12 02:37:32 on dist-test-slave-mn4r"
server_key: "322dd2f9226f570d487955d6bb3bb105"
server_key_iv: "403b7eb5f09b4d0306b44170faa48e06"
server_key_version: "encryptionkey@0"
I20260812 02:37:32.862321 19175 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502156971023-7698-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502156971023-7698-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502156971023-7698-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:37:32.875790 19175 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:37:32.876598 19175 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:37:32.876856 19175 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:37:32.877759 19175 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:37:32.879599 19175 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:37:32.879689 19175 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:37:32.879776 19175 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:37:32.879830 19175 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:37:32.914583 19175 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.129:44547
I20260812 02:37:32.914644 19334 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.129:44547 every 8 connection(s)
I20260812 02:37:32.915987 19175 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502156971023-7698-0/minicluster-data/ts-0/data/info.pb
I20260812 02:37:32.919791 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 19175
I20260812 02:37:32.920006 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502156971023-7698-0/minicluster-data/ts-0/wal/instance
I20260812 02:37:32.920917 7698 external_mini_cluster.cc:1428] Setting key 1807f8d308457d2762537ffc91119b2f
I20260812 02:37:32.923811 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502156971023-7698-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502156971023-7698-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502156971023-7698-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502156971023-7698-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.130:0
--local_ip_for_outbound_sockets=127.7.132.130
--webserver_interface=127.7.132.130
--webserver_port=0
--tserver_master_addrs=127.7.132.190:38469
--builtin_ntp_servers=127.7.132.148:33365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260812 02:37:32.929903 19337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:32.919044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:51175 (local address 127.7.132.190:38469)
0812 02:37:32.919569 (+ 525us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:32.919574 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:32.920999 (+ 1425us) server_negotiation.cc:407] Connection header received
0812 02:37:32.923225 (+ 2226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:32.923230 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:32.923323 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:32.923449 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:37:32.924622 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:32.925344 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:32.926999 (+ 1655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:32.927409 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:32.927555 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:32.928976 (+ 1421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:37:32.928982 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:37:32.928987 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:37:32.929009 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:37:32.929051 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:37:32.929064 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:37:32.929171 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0812 02:37:32.929412 (+ 241us) server_negotiation.cc:299] Negotiation successful
0812 02:37:32.929525 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":383,"thread_start_us":278,"threads_started":1}
I20260812 02:37:32.931061 19338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:32.919819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:38469 (local address 127.7.132.129:51175)
0812 02:37:32.920829 (+ 1010us) negotiation.cc:107] Waiting for socket to connect
0812 02:37:32.920873 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 02:37:32.922942 (+ 2069us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:37:32.923702 (+ 760us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:37:32.923711 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:37:32.923803 (+ 92us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:37:32.924455 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:32.924472 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:32.925517 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:32.925522 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:32.926845 (+ 1323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:32.926857 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:32.927641 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:32.927649 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:32.927883 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:32.928624 (+ 741us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 02:37:32.928627 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:37:32.928668 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 02:37:32.928748 (+ 80us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:37:32.929172 (+ 424us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:37:32.929186 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:37:32.929196 (+ 10us) client_negotiation.cc:769] Sending connection context
0812 02:37:32.929386 (+ 190us) client_negotiation.cc:240] Negotiation successful
0812 02:37:32.929788 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":533,"thread_start_us":321,"threads_started":1}
I20260812 02:37:32.932844 19335 heartbeater.cc:347] Connected to a master server at 127.7.132.190:38469
I20260812 02:37:32.933176 19335 heartbeater.cc:464] Registering TS with master...
I20260812 02:37:32.933972 19335 heartbeater.cc:511] Master 127.7.132.190:38469 requested a full tablet report, sending...
I20260812 02:37:32.936162 19102 ts_manager.cc:194] Registered new tserver with Master: a57f4281d746461cbe72884b8311a6f2 (127.7.132.129:44547)
I20260812 02:37:32.937683 19102 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.132.129:51175
W20260812 02:37:33.263484 19339 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:37:33.263777 19339 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:37:33.263928 19339 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:37:33.271431 19339 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:37:33.271530 19339 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:37:33.271709 19339 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.130
I20260812 02:37:33.284464 19339 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:33365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502156971023-7698-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502156971023-7698-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502156971023-7698-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.132.130
--webserver_port=0
--tserver_master_addrs=127.7.132.190:38469
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.130
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502156971023-7698-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:37:33.285894 19339 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:37:33.287752 19339 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:37:33.301342 19349 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:37:33.301384 19347 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:37:33.301307 19345 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:37:33.302299 19339 server_base.cc:1034] running on GCE node
I20260812 02:37:33.302851 19339 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:37:33.303740 19339 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:37:33.304939 19339 hybrid_clock.cc:648] HybridClock initialized: now 1786502253304879 us; error 61 us; skew 500 ppm
I20260812 02:37:33.305202 19339 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 02:37:33.307596 19339 webserver.cc:460] Webserver started at http://127.7.132.130:41765/ using document root <none> and password file <none>
I20260812 02:37:33.308305 19339 fs_manager.cc:359] Metadata directory not provided
I20260812 02:37:33.308393 19339 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:37:33.308753 19339 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:37:33.311619 19339 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502156971023-7698-0/minicluster-data/ts-1/data/instance:
uuid: "39e14701efee4a129a8ab7708886b82e"
format_stamp: "Formatted at 2026-08-12 02:37:33 on dist-test-slave-mn4r"
server_key: "b176afe017eb6ebdcc4ad350c2e8d2b2"
server_key_iv: "e12fd9815f1c764ef402bac516b91cd1"
server_key_version: "encryptionkey@0"
I20260812 02:37:33.312453 19339 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502156971023-7698-0/minicluster-data/ts-1/wal/instance:
uuid: "39e14701efee4a129a8ab7708886b82e"
format_stamp: "Formatted at 2026-08-12 02:37:33 on dist-test-slave-mn4r"
server_key: "b176afe017eb6ebdcc4ad350c2e8d2b2"
server_key_iv: "e12fd9815f1c764ef402bac516b91cd1"
server_key_version: "encryptionkey@0"
I20260812 02:37:33.317579 19339 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260812 02:37:33.321249 19356 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:37:33.322650 19339 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 02:37:33.322834 19339 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502156971023-7698-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502156971023-7698-0/minicluster-data/ts-1/wal
uuid: "39e14701efee4a129a8ab7708886b82e"
format_stamp: "Formatted at 2026-08-12 02:37:33 on dist-test-slave-mn4r"
server_key: "b176afe017eb6ebdcc4ad350c2e8d2b2"
server_key_iv: "e12fd9815f1c764ef402bac516b91cd1"
server_key_version: "encryptionkey@0"
I20260812 02:37:33.323060 19339 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502156971023-7698-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502156971023-7698-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502156971023-7698-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:37:33.343253 19339 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:37:33.344447 19339 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:37:33.344873 19339 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:37:33.346390 19339 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:37:33.348487 19339 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:37:33.348614 19339 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:37:33.348727 19339 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:37:33.348815 19339 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:37:33.384452 19339 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.130:40989
I20260812 02:37:33.384522 19492 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.130:40989 every 8 connection(s)
I20260812 02:37:33.385856 19339 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502156971023-7698-0/minicluster-data/ts-1/data/info.pb
I20260812 02:37:33.393402 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 19339
I20260812 02:37:33.393571 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502156971023-7698-0/minicluster-data/ts-1/wal/instance
I20260812 02:37:33.394157 7698 external_mini_cluster.cc:1428] Setting key 9b5c85ca3dc14497e660f97ae8c2f898
I20260812 02:37:33.396893 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502156971023-7698-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502156971023-7698-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502156971023-7698-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502156971023-7698-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.131:0
--local_ip_for_outbound_sockets=127.7.132.131
--webserver_interface=127.7.132.131
--webserver_port=0
--tserver_master_addrs=127.7.132.190:38469
--builtin_ntp_servers=127.7.132.148:33365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260812 02:37:33.399288 19337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:33.388644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:50979 (local address 127.7.132.190:38469)
0812 02:37:33.388843 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:33.388848 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:33.390371 (+ 1523us) server_negotiation.cc:407] Connection header received
0812 02:37:33.392474 (+ 2103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:33.392481 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:33.392603 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:33.392723 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:37:33.393912 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:33.394797 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:33.396384 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:33.396804 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:33.396939 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:33.398457 (+ 1518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:37:33.398462 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:37:33.398466 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:37:33.398482 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:37:33.398514 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:37:33.398526 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:37:33.398632 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0812 02:37:33.398856 (+ 224us) server_negotiation.cc:299] Negotiation successful
0812 02:37:33.398957 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260812 02:37:33.400317 19495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:33.389246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:38469 (local address 127.7.132.130:50979)
0812 02:37:33.390212 (+ 966us) negotiation.cc:107] Waiting for socket to connect
0812 02:37:33.390249 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 02:37:33.392171 (+ 1922us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:37:33.392959 (+ 788us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:37:33.392967 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:37:33.393062 (+ 95us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:37:33.393736 (+ 674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:33.393757 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:33.394965 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:33.394971 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:33.396234 (+ 1263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:33.396246 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:33.397052 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:33.397061 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:33.397322 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:33.398131 (+ 809us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 02:37:33.398134 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:37:33.398177 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 02:37:33.398262 (+ 85us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:37:33.398632 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:37:33.398646 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:37:33.398656 (+ 10us) client_negotiation.cc:769] Sending connection context
0812 02:37:33.398840 (+ 184us) client_negotiation.cc:240] Negotiation successful
0812 02:37:33.399249 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":291,"threads_started":1}
I20260812 02:37:33.401731 19493 heartbeater.cc:347] Connected to a master server at 127.7.132.190:38469
I20260812 02:37:33.402096 19493 heartbeater.cc:464] Registering TS with master...
I20260812 02:37:33.403056 19493 heartbeater.cc:511] Master 127.7.132.190:38469 requested a full tablet report, sending...
I20260812 02:37:33.404697 19102 ts_manager.cc:194] Registered new tserver with Master: 39e14701efee4a129a8ab7708886b82e (127.7.132.130:40989)
I20260812 02:37:33.405958 19102 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.132.130:50979
W20260812 02:37:33.737218 19496 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:37:33.737483 19496 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:37:33.737644 19496 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:37:33.744764 19496 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:37:33.744858 19496 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:37:33.745078 19496 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.131
I20260812 02:37:33.754334 19496 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:33365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502156971023-7698-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502156971023-7698-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502156971023-7698-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.132.131
--webserver_port=0
--tserver_master_addrs=127.7.132.190:38469
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.131
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502156971023-7698-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:37:33.755692 19496 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:37:33.757761 19496 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:37:33.771695 19501 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:37:33.771656 19504 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:37:33.771744 19502 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:37:33.775589 19496 server_base.cc:1034] running on GCE node
I20260812 02:37:33.776263 19496 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:37:33.777288 19496 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:37:33.778537 19496 hybrid_clock.cc:648] HybridClock initialized: now 1786502253778484 us; error 46 us; skew 500 ppm
I20260812 02:37:33.778872 19496 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 02:37:33.781273 19496 webserver.cc:460] Webserver started at http://127.7.132.131:37891/ using document root <none> and password file <none>
I20260812 02:37:33.782020 19496 fs_manager.cc:359] Metadata directory not provided
I20260812 02:37:33.782182 19496 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:37:33.782536 19496 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:37:33.785495 19496 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502156971023-7698-0/minicluster-data/ts-2/data/instance:
uuid: "3e00ad14825b4f8b93ecc6286b1c456d"
format_stamp: "Formatted at 2026-08-12 02:37:33 on dist-test-slave-mn4r"
server_key: "e6670cc99a59043c32004846324776db"
server_key_iv: "d90686e5be349e4ee50513486a303803"
server_key_version: "encryptionkey@0"
I20260812 02:37:33.786417 19496 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502156971023-7698-0/minicluster-data/ts-2/wal/instance:
uuid: "3e00ad14825b4f8b93ecc6286b1c456d"
format_stamp: "Formatted at 2026-08-12 02:37:33 on dist-test-slave-mn4r"
server_key: "e6670cc99a59043c32004846324776db"
server_key_iv: "d90686e5be349e4ee50513486a303803"
server_key_version: "encryptionkey@0"
I20260812 02:37:33.791800 19496 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260812 02:37:33.795384 19514 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:37:33.796862 19496 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 02:37:33.797036 19496 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502156971023-7698-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502156971023-7698-0/minicluster-data/ts-2/wal
uuid: "3e00ad14825b4f8b93ecc6286b1c456d"
format_stamp: "Formatted at 2026-08-12 02:37:33 on dist-test-slave-mn4r"
server_key: "e6670cc99a59043c32004846324776db"
server_key_iv: "d90686e5be349e4ee50513486a303803"
server_key_version: "encryptionkey@0"
I20260812 02:37:33.797260 19496 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502156971023-7698-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502156971023-7698-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502156971023-7698-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:37:33.825342 19496 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:37:33.826335 19496 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:37:33.826659 19496 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:37:33.827693 19496 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:37:33.829133 19496 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:37:33.829221 19496 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:37:33.829301 19496 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:37:33.829360 19496 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:37:33.867326 19496 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.131:45791
I20260812 02:37:33.867376 19654 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.131:45791 every 8 connection(s)
I20260812 02:37:33.868705 19496 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502156971023-7698-0/minicluster-data/ts-2/data/info.pb
I20260812 02:37:33.875744 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 19496
I20260812 02:37:33.875905 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786502156971023-7698-0/minicluster-data/ts-2/wal/instance
I20260812 02:37:33.876410 7698 external_mini_cluster.cc:1428] Setting key cc4d26e3b0732e16182a626c186d5cf1
I20260812 02:37:33.881793 19337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:33.871341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:46927 (local address 127.7.132.190:38469)
0812 02:37:33.871539 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:33.871543 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:33.872961 (+ 1418us) server_negotiation.cc:407] Connection header received
0812 02:37:33.874862 (+ 1901us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:33.874868 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:33.874962 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:33.875072 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:37:33.876229 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:33.877044 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:33.879065 (+ 2021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:33.879745 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:33.879885 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:33.881120 (+ 1235us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:37:33.881125 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:37:33.881128 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:37:33.881147 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:37:33.881179 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:37:33.881190 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:37:33.881263 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 02:37:33.881456 (+ 193us) server_negotiation.cc:299] Negotiation successful
0812 02:37:33.881538 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260812 02:37:33.882815 19657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:33.871911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:38469 (local address 127.7.132.131:46927)
0812 02:37:33.872807 (+ 896us) negotiation.cc:107] Waiting for socket to connect
0812 02:37:33.872850 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 02:37:33.874585 (+ 1735us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:37:33.875295 (+ 710us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:37:33.875303 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:37:33.875393 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:37:33.876058 (+ 665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:33.876079 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:33.877230 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:33.877234 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:33.878895 (+ 1661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:33.878913 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:33.879949 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:33.879954 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:33.880154 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:33.880815 (+ 661us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 02:37:33.880818 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:37:33.880857 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 02:37:33.880935 (+ 78us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:37:33.881276 (+ 341us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:37:33.881284 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:37:33.881290 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 02:37:33.881433 (+ 143us) client_negotiation.cc:240] Negotiation successful
0812 02:37:33.881726 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":269,"threads_started":1}
I20260812 02:37:33.884218 19655 heartbeater.cc:347] Connected to a master server at 127.7.132.190:38469
I20260812 02:37:33.884516 19655 heartbeater.cc:464] Registering TS with master...
I20260812 02:37:33.885250 19655 heartbeater.cc:511] Master 127.7.132.190:38469 requested a full tablet report, sending...
I20260812 02:37:33.886653 19102 ts_manager.cc:194] Registered new tserver with Master: 3e00ad14825b4f8b93ecc6286b1c456d (127.7.132.131:45791)
I20260812 02:37:33.887429 19102 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.132.131:46927
I20260812 02:37:33.891793 7698 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260812 02:37:33.898008 19337 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:37:33.896332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55906 (local address 127.7.132.190:38469)
0812 02:37:33.896596 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:33.896606 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:33.896992 (+ 386us) server_negotiation.cc:407] Connection header received
0812 02:37:33.897167 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:33.897171 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:33.897248 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:33.897337 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:37:33.897809 (+ 472us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55906: BlockingRecv error: recv got EOF from 127.0.0.1:55906 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 02:37:33.898680 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 19175
I20260812 02:37:33.906388 19667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:33.899637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49842 (local address 127.7.132.129:44547)
0812 02:37:33.900208 (+ 571us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:33.900214 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:33.900240 (+ 26us) server_negotiation.cc:407] Connection header received
0812 02:37:33.900300 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:33.900306 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:33.900499 (+ 193us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:33.900641 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:37:33.901005 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:33.902175 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:33.903416 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:33.903939 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:33.904038 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:33.904992 (+ 954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:37:33.905000 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:37:33.905012 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:37:33.905589 (+ 577us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:37:33.905637 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:37:33.905661 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:37:33.905749 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0812 02:37:33.905870 (+ 121us) server_negotiation.cc:299] Negotiation successful
0812 02:37:33.906022 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":422,"thread_start_us":299,"threads_started":1}
I20260812 02:37:33.907541 19327 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:37:33.990021 19335 heartbeater.cc:503] Master 127.7.132.190:38469 was elected leader, sending a full tablet report...
I20260812 02:37:33.991882 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 19175
I20260812 02:37:34.012914 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 19339
I20260812 02:37:34.021119 19669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:34.013885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47676 (local address 127.7.132.130:40989)
0812 02:37:34.014459 (+ 574us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:34.014466 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:34.014495 (+ 29us) server_negotiation.cc:407] Connection header received
0812 02:37:34.014554 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:34.014560 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:34.014775 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:34.014952 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:37:34.015374 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:34.016361 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:34.017623 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:34.018298 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:34.018435 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:34.019533 (+ 1098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:37:34.019545 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:37:34.019564 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:37:34.020262 (+ 698us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:37:34.020311 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:37:34.020331 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:37:34.020460 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0812 02:37:34.020645 (+ 185us) server_negotiation.cc:299] Negotiation successful
0812 02:37:34.020776 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":434,"thread_start_us":310,"threads_started":1}
I20260812 02:37:34.022531 19488 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:37:34.105907 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 19339
I20260812 02:37:34.125202 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 19496
I20260812 02:37:34.133774 19671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:34.126302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35330 (local address 127.7.132.131:45791)
0812 02:37:34.127002 (+ 700us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:34.127010 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:34.127042 (+ 32us) server_negotiation.cc:407] Connection header received
0812 02:37:34.127105 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:34.127111 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:34.127333 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:34.127521 (+ 188us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:37:34.128038 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:34.129103 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:34.130430 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:34.131022 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:34.131168 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:34.132268 (+ 1100us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:37:34.132277 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:37:34.132290 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:37:34.132899 (+ 609us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:37:34.132946 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:37:34.132966 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:37:34.133121 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0812 02:37:34.133268 (+ 147us) server_negotiation.cc:299] Negotiation successful
0812 02:37:34.133399 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":526,"thread_start_us":368,"threads_started":1}
I20260812 02:37:34.135193 19647 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:37:34.218137 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 19496
I20260812 02:37:34.239845 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 19063
I20260812 02:37:34.241183 19140 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:37:34.346657 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 19063
2026-08-12T02:37:34Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (2566 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-12T02:37:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T02:37:34Z Disabled control of system clock
I20260812 02:37:34.433746 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502156971023-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502156971023-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502156971023-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:35737
--webserver_interface=127.7.132.190
--webserver_port=0
--builtin_ntp_servers=127.7.132.148:34961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.132.190:35737
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260812 02:37:34.779991 19678 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:37:34.780277 19678 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:37:34.780457 19678 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:37:34.787918 19678 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:37:34.788031 19678 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:37:34.788110 19678 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:37:34.788218 19678 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:37:34.788297 19678 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:37:34.798038 19678 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:34961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502156971023-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502156971023-7698-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.132.190:35737
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:35737
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.132.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502156971023-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:37:34.799556 19678 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:37:34.801510 19678 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:37:34.814260 19687 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:37:34.814193 19689 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:37:34.814301 19685 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:37:34.815124 19678 server_base.cc:1034] running on GCE node
I20260812 02:37:34.815925 19678 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:37:34.817332 19678 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:37:34.818574 19678 hybrid_clock.cc:648] HybridClock initialized: now 1786502254818525 us; error 42 us; skew 500 ppm
I20260812 02:37:34.818951 19678 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 02:37:34.821477 19678 webserver.cc:460] Webserver started at http://127.7.132.190:43287/ using document root <none> and password file <none>
I20260812 02:37:34.822404 19678 fs_manager.cc:359] Metadata directory not provided
I20260812 02:37:34.822548 19678 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:37:34.822888 19678 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:37:34.828060 19678 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502156971023-7698-0/minicluster-data/master-0/data/instance:
uuid: "405569331c274859b64418b0f73dac70"
format_stamp: "Formatted at 2026-08-12 02:37:34 on dist-test-slave-mn4r"
server_key: "b3f4a6718934b36b2c050958bcee5a0f"
server_key_iv: "31b50598c0e59938d7a1a08984345615"
server_key_version: "encryptionkey@0"
I20260812 02:37:34.828908 19678 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502156971023-7698-0/minicluster-data/master-0/wal/instance:
uuid: "405569331c274859b64418b0f73dac70"
format_stamp: "Formatted at 2026-08-12 02:37:34 on dist-test-slave-mn4r"
server_key: "b3f4a6718934b36b2c050958bcee5a0f"
server_key_iv: "31b50598c0e59938d7a1a08984345615"
server_key_version: "encryptionkey@0"
I20260812 02:37:34.834378 19678 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.004s
I20260812 02:37:34.837924 19697 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:37:34.839385 19678 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 02:37:34.839625 19678 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502156971023-7698-0/minicluster-data/master-0/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502156971023-7698-0/minicluster-data/master-0/wal
uuid: "405569331c274859b64418b0f73dac70"
format_stamp: "Formatted at 2026-08-12 02:37:34 on dist-test-slave-mn4r"
server_key: "b3f4a6718934b36b2c050958bcee5a0f"
server_key_iv: "31b50598c0e59938d7a1a08984345615"
server_key_version: "encryptionkey@0"
I20260812 02:37:34.839823 19678 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502156971023-7698-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502156971023-7698-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502156971023-7698-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:37:34.845722 19678 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:37:34.846534 19678 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:37:34.846786 19678 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:37:34.865664 19678 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.190:35737
I20260812 02:37:34.865715 19759 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.190:35737 every 8 connection(s)
I20260812 02:37:34.867141 19678 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
I20260812 02:37:34.870476 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 19678
I20260812 02:37:34.870720 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502156971023-7698-0/minicluster-data/master-0/wal/instance
I20260812 02:37:34.871136 19760 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:37:34.871462 7698 external_mini_cluster.cc:1428] Setting key 99de8c5ba31e9941062f237296c47025
I20260812 02:37:34.879751 19763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:34.873489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33986 (local address 127.7.132.190:35737)
0812 02:37:34.874593 (+ 1104us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:34.874604 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:34.874648 (+ 44us) server_negotiation.cc:407] Connection header received
0812 02:37:34.876132 (+ 1484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:34.876153 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:34.876580 (+ 427us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:34.876908 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:37:34.878127 (+ 1219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:37:34.878139 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:37:34.878153 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:37:34.878172 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:37:34.878240 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:37:34.878257 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:37:34.878439 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0812 02:37:34.878578 (+ 139us) server_negotiation.cc:299] Negotiation successful
0812 02:37:34.878875 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":652,"thread_start_us":429,"threads_started":1}
I20260812 02:37:34.884830 19760 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 405569331c274859b64418b0f73dac70: Bootstrap starting.
I20260812 02:37:34.888017 19760 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 405569331c274859b64418b0f73dac70: Neither blocks nor log segments found. Creating new log.
I20260812 02:37:34.889026 19760 log.cc:824] T 00000000000000000000000000000000 P 405569331c274859b64418b0f73dac70: Log is configured to *not* fsync() on all Append() calls
I20260812 02:37:34.891700 19760 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 405569331c274859b64418b0f73dac70: No bootstrap required, opened a new log
I20260812 02:37:34.899354 19760 raft_consensus.cc:348] T 00000000000000000000000000000000 P 405569331c274859b64418b0f73dac70 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "405569331c274859b64418b0f73dac70" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 35737 } }
I20260812 02:37:34.899791 19760 raft_consensus.cc:374] T 00000000000000000000000000000000 P 405569331c274859b64418b0f73dac70 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:37:34.899914 19760 raft_consensus.cc:729] T 00000000000000000000000000000000 P 405569331c274859b64418b0f73dac70 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 405569331c274859b64418b0f73dac70, State: Initialized, Role: FOLLOWER
I20260812 02:37:34.900542 19760 consensus_queue.cc:260] T 00000000000000000000000000000000 P 405569331c274859b64418b0f73dac70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "405569331c274859b64418b0f73dac70" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 35737 } }
I20260812 02:37:34.900826 19760 raft_consensus.cc:388] T 00000000000000000000000000000000 P 405569331c274859b64418b0f73dac70 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 02:37:34.900995 19760 raft_consensus.cc:482] T 00000000000000000000000000000000 P 405569331c274859b64418b0f73dac70 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 02:37:34.901240 19760 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 405569331c274859b64418b0f73dac70 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:37:34.903746 19760 raft_consensus.cc:504] T 00000000000000000000000000000000 P 405569331c274859b64418b0f73dac70 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "405569331c274859b64418b0f73dac70" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 35737 } }
I20260812 02:37:34.904299 19760 leader_election.cc:302] T 00000000000000000000000000000000 P 405569331c274859b64418b0f73dac70 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 405569331c274859b64418b0f73dac70; no voters:
I20260812 02:37:34.904996 19760 leader_election.cc:288] T 00000000000000000000000000000000 P 405569331c274859b64418b0f73dac70 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 02:37:34.905347 19765 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 405569331c274859b64418b0f73dac70 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:37:34.906117 19765 raft_consensus.cc:686] T 00000000000000000000000000000000 P 405569331c274859b64418b0f73dac70 [term 1 LEADER]: Becoming Leader. State: Replica: 405569331c274859b64418b0f73dac70, State: Running, Role: LEADER
I20260812 02:37:34.906715 19765 consensus_queue.cc:237] T 00000000000000000000000000000000 P 405569331c274859b64418b0f73dac70 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "405569331c274859b64418b0f73dac70" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 35737 } }
I20260812 02:37:34.907513 19760 sys_catalog.cc:564] T 00000000000000000000000000000000 P 405569331c274859b64418b0f73dac70 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:37:34.909574 19766 sys_catalog.cc:455] T 00000000000000000000000000000000 P 405569331c274859b64418b0f73dac70 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "405569331c274859b64418b0f73dac70" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "405569331c274859b64418b0f73dac70" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 35737 } } }
I20260812 02:37:34.910061 19766 sys_catalog.cc:458] T 00000000000000000000000000000000 P 405569331c274859b64418b0f73dac70 [sys.catalog]: This master's current role is: LEADER
I20260812 02:37:34.914134 19768 sys_catalog.cc:455] T 00000000000000000000000000000000 P 405569331c274859b64418b0f73dac70 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 405569331c274859b64418b0f73dac70. Latest consensus state: current_term: 1 leader_uuid: "405569331c274859b64418b0f73dac70" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "405569331c274859b64418b0f73dac70" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 35737 } } }
I20260812 02:37:34.914584 19768 sys_catalog.cc:458] T 00000000000000000000000000000000 P 405569331c274859b64418b0f73dac70 [sys.catalog]: This master's current role is: LEADER
I20260812 02:37:34.914580 19773 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:37:34.920274 19773 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 02:37:34.929489 19773 catalog_manager.cc:1347] Generated new cluster ID: 00c4d5a2b834408289df27b0d9d8f566
I20260812 02:37:34.929620 19773 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 02:37:34.951879 19773 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 02:37:34.952385 19773 catalog_manager.cc:1504] Loading token signing keys...
I20260812 02:37:34.959882 19773 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 405569331c274859b64418b0f73dac70: Generated new TSK 0
I20260812 02:37:34.960685 19773 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 02:37:34.968508 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502156971023-7698-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502156971023-7698-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502156971023-7698-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502156971023-7698-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.129:0
--local_ip_for_outbound_sockets=127.7.132.129
--webserver_interface=127.7.132.129
--webserver_port=0
--tserver_master_addrs=127.7.132.190:35737
--builtin_ntp_servers=127.7.132.148:34961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260812 02:37:35.317492 19792 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:37:35.317782 19792 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:37:35.317956 19792 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:37:35.325304 19792 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:37:35.325410 19792 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:37:35.325598 19792 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.129
I20260812 02:37:35.334668 19792 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:34961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502156971023-7698-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502156971023-7698-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502156971023-7698-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.132.129
--webserver_port=0
--tserver_master_addrs=127.7.132.190:35737
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.129
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502156971023-7698-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:37:35.336308 19792 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:37:35.338305 19792 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:37:35.354379 19800 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:37:35.354439 19802 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:37:35.355523 19799 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:37:35.356734 19792 server_base.cc:1034] running on GCE node
I20260812 02:37:35.357355 19792 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:37:35.358450 19792 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:37:35.359690 19792 hybrid_clock.cc:648] HybridClock initialized: now 1786502255359645 us; error 34 us; skew 500 ppm
I20260812 02:37:35.359980 19792 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 02:37:35.362356 19792 webserver.cc:460] Webserver started at http://127.7.132.129:32979/ using document root <none> and password file <none>
I20260812 02:37:35.363070 19792 fs_manager.cc:359] Metadata directory not provided
I20260812 02:37:35.363160 19792 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:37:35.363442 19792 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:37:35.366379 19792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502156971023-7698-0/minicluster-data/ts-0/data/instance:
uuid: "39cb0c12c14f4d35a09e7bb390332d6f"
format_stamp: "Formatted at 2026-08-12 02:37:35 on dist-test-slave-mn4r"
server_key: "8b38313f16514c917bbee757ff97b784"
server_key_iv: "dd09d57796296709c7ebc0428e1a2568"
server_key_version: "encryptionkey@0"
I20260812 02:37:35.367341 19792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502156971023-7698-0/minicluster-data/ts-0/wal/instance:
uuid: "39cb0c12c14f4d35a09e7bb390332d6f"
format_stamp: "Formatted at 2026-08-12 02:37:35 on dist-test-slave-mn4r"
server_key: "8b38313f16514c917bbee757ff97b784"
server_key_iv: "dd09d57796296709c7ebc0428e1a2568"
server_key_version: "encryptionkey@0"
I20260812 02:37:35.372862 19792 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260812 02:37:35.376327 19809 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:37:35.377725 19792 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 02:37:35.377863 19792 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502156971023-7698-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502156971023-7698-0/minicluster-data/ts-0/wal
uuid: "39cb0c12c14f4d35a09e7bb390332d6f"
format_stamp: "Formatted at 2026-08-12 02:37:35 on dist-test-slave-mn4r"
server_key: "8b38313f16514c917bbee757ff97b784"
server_key_iv: "dd09d57796296709c7ebc0428e1a2568"
server_key_version: "encryptionkey@0"
I20260812 02:37:35.378237 19792 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502156971023-7698-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502156971023-7698-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502156971023-7698-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:37:35.383882 19792 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:37:35.384618 19792 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:37:35.384855 19792 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:37:35.385723 19792 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:37:35.387166 19792 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:37:35.387255 19792 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:37:35.387336 19792 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:37:35.387391 19792 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:37:35.420843 19792 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.129:36701
I20260812 02:37:35.420904 19951 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.129:36701 every 8 connection(s)
I20260812 02:37:35.422333 19792 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502156971023-7698-0/minicluster-data/ts-0/data/info.pb
I20260812 02:37:35.429407 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 19792
I20260812 02:37:35.429653 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502156971023-7698-0/minicluster-data/ts-0/wal/instance
I20260812 02:37:35.430568 7698 external_mini_cluster.cc:1428] Setting key a1121b153c7b66bb5194cd7dd5bd9dae
I20260812 02:37:35.431748 19954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:35.424951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:55977 (local address 127.7.132.190:35737)
0812 02:37:35.425420 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:35.425426 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:35.427145 (+ 1719us) server_negotiation.cc:407] Connection header received
0812 02:37:35.428862 (+ 1717us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:35.428867 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:35.428925 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:35.429030 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:37:35.430892 (+ 1862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:37:35.430897 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:37:35.430901 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:37:35.430920 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:37:35.430977 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:37:35.430991 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:37:35.431110 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0812 02:37:35.431396 (+ 286us) server_negotiation.cc:299] Negotiation successful
0812 02:37:35.431476 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":225,"threads_started":1}
I20260812 02:37:35.432845 19957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:35.425625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:35737 (local address 127.7.132.129:55977)
0812 02:37:35.426976 (+ 1351us) negotiation.cc:107] Waiting for socket to connect
0812 02:37:35.427022 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 02:37:35.428562 (+ 1540us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:37:35.429340 (+ 778us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:37:35.429352 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:37:35.429483 (+ 131us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:37:35.430473 (+ 990us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 02:37:35.430477 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:37:35.430543 (+ 66us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 02:37:35.430675 (+ 132us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:37:35.431110 (+ 435us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:37:35.431122 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:37:35.431132 (+ 10us) client_negotiation.cc:769] Sending connection context
0812 02:37:35.431406 (+ 274us) client_negotiation.cc:240] Negotiation successful
0812 02:37:35.431679 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":748,"thread_start_us":480,"threads_started":1}
I20260812 02:37:35.433915 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502156971023-7698-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502156971023-7698-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502156971023-7698-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502156971023-7698-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.130:0
--local_ip_for_outbound_sockets=127.7.132.130
--webserver_interface=127.7.132.130
--webserver_port=0
--tserver_master_addrs=127.7.132.190:35737
--builtin_ntp_servers=127.7.132.148:34961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260812 02:37:35.434406 19952 heartbeater.cc:347] Connected to a master server at 127.7.132.190:35737
I20260812 02:37:35.434686 19952 heartbeater.cc:464] Registering TS with master...
I20260812 02:37:35.435218 19952 heartbeater.cc:511] Master 127.7.132.190:35737 requested a full tablet report, sending...
I20260812 02:37:35.437287 19716 ts_manager.cc:194] Registered new tserver with Master: 39cb0c12c14f4d35a09e7bb390332d6f (127.7.132.129:36701)
W20260812 02:37:35.781512 19958 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:37:35.781862 19958 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:37:35.782125 19958 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:37:35.789538 19958 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:37:35.789646 19958 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:37:35.789829 19958 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.130
I20260812 02:37:35.799295 19958 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:34961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502156971023-7698-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502156971023-7698-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502156971023-7698-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.132.130
--webserver_port=0
--tserver_master_addrs=127.7.132.190:35737
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.130
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502156971023-7698-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:37:35.800717 19958 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:37:35.802700 19958 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:37:35.816874 19965 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:37:35.816964 19968 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:37:35.817008 19966 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:37:35.817679 19958 server_base.cc:1034] running on GCE node
I20260812 02:37:35.818439 19958 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:37:35.819442 19958 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:37:35.820680 19958 hybrid_clock.cc:648] HybridClock initialized: now 1786502255820635 us; error 60 us; skew 500 ppm
I20260812 02:37:35.821041 19958 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 02:37:35.827351 19958 webserver.cc:460] Webserver started at http://127.7.132.130:32801/ using document root <none> and password file <none>
I20260812 02:37:35.828104 19958 fs_manager.cc:359] Metadata directory not provided
I20260812 02:37:35.828243 19958 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:37:35.828596 19958 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:37:35.831661 19958 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502156971023-7698-0/minicluster-data/ts-1/data/instance:
uuid: "87863cf5c9db40e3b2e206803b3b8df5"
format_stamp: "Formatted at 2026-08-12 02:37:35 on dist-test-slave-mn4r"
server_key: "c6018d21fa650fbb0098c233c519a16f"
server_key_iv: "adf9d493897a5785368013e2e6b644b2"
server_key_version: "encryptionkey@0"
I20260812 02:37:35.832505 19958 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502156971023-7698-0/minicluster-data/ts-1/wal/instance:
uuid: "87863cf5c9db40e3b2e206803b3b8df5"
format_stamp: "Formatted at 2026-08-12 02:37:35 on dist-test-slave-mn4r"
server_key: "c6018d21fa650fbb0098c233c519a16f"
server_key_iv: "adf9d493897a5785368013e2e6b644b2"
server_key_version: "encryptionkey@0"
I20260812 02:37:35.837797 19958 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260812 02:37:35.841554 19978 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:37:35.843115 19958 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260812 02:37:35.843307 19958 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502156971023-7698-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502156971023-7698-0/minicluster-data/ts-1/wal
uuid: "87863cf5c9db40e3b2e206803b3b8df5"
format_stamp: "Formatted at 2026-08-12 02:37:35 on dist-test-slave-mn4r"
server_key: "c6018d21fa650fbb0098c233c519a16f"
server_key_iv: "adf9d493897a5785368013e2e6b644b2"
server_key_version: "encryptionkey@0"
I20260812 02:37:35.843572 19958 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502156971023-7698-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502156971023-7698-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502156971023-7698-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:37:35.849860 19958 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:37:35.850863 19958 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:37:35.851153 19958 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:37:35.852136 19958 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:37:35.853577 19958 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:37:35.853704 19958 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:37:35.853833 19958 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:37:35.853921 19958 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:37:35.894726 19958 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.130:43717
I20260812 02:37:35.894766 20112 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.130:43717 every 8 connection(s)
I20260812 02:37:35.896226 19958 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502156971023-7698-0/minicluster-data/ts-1/data/info.pb
I20260812 02:37:35.903986 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 19958
I20260812 02:37:35.904188 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502156971023-7698-0/minicluster-data/ts-1/wal/instance
I20260812 02:37:35.904680 7698 external_mini_cluster.cc:1428] Setting key ec2ba70bd04f25912ab2e819ef338b45
I20260812 02:37:35.904860 19954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:35.898991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:48017 (local address 127.7.132.190:35737)
0812 02:37:35.899184 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:35.899189 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:35.900644 (+ 1455us) server_negotiation.cc:407] Connection header received
0812 02:37:35.902512 (+ 1868us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:35.902518 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:35.902615 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:35.902734 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:37:35.904062 (+ 1328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:37:35.904070 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:37:35.904076 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:37:35.904098 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:37:35.904138 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:37:35.904155 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:37:35.904298 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0812 02:37:35.904497 (+ 199us) server_negotiation.cc:299] Negotiation successful
0812 02:37:35.904569 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260812 02:37:35.905824 20117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:35.899530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:35737 (local address 127.7.132.130:48017)
0812 02:37:35.900465 (+ 935us) negotiation.cc:107] Waiting for socket to connect
0812 02:37:35.900501 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 02:37:35.902210 (+ 1709us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:37:35.902980 (+ 770us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:37:35.902989 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:37:35.903081 (+ 92us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:37:35.903717 (+ 636us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 02:37:35.903720 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:37:35.903759 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 02:37:35.903845 (+ 86us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:37:35.904265 (+ 420us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:37:35.904277 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:37:35.904285 (+ 8us) client_negotiation.cc:769] Sending connection context
0812 02:37:35.904600 (+ 315us) client_negotiation.cc:240] Negotiation successful
0812 02:37:35.904892 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":278,"threads_started":1}
I20260812 02:37:35.907224 20113 heartbeater.cc:347] Connected to a master server at 127.7.132.190:35737
I20260812 02:37:35.907547 20113 heartbeater.cc:464] Registering TS with master...
I20260812 02:37:35.907718 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502156971023-7698-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502156971023-7698-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502156971023-7698-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502156971023-7698-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.131:0
--local_ip_for_outbound_sockets=127.7.132.131
--webserver_interface=127.7.132.131
--webserver_port=0
--tserver_master_addrs=127.7.132.190:35737
--builtin_ntp_servers=127.7.132.148:34961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260812 02:37:35.908191 20113 heartbeater.cc:511] Master 127.7.132.190:35737 requested a full tablet report, sending...
I20260812 02:37:35.909610 19716 ts_manager.cc:194] Registered new tserver with Master: 87863cf5c9db40e3b2e206803b3b8df5 (127.7.132.130:43717)
W20260812 02:37:36.257908 20118 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:37:36.258311 20118 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:37:36.258488 20118 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:37:36.265923 20118 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:37:36.266032 20118 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:37:36.266228 20118 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.131
I20260812 02:37:36.275344 20118 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:34961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502156971023-7698-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502156971023-7698-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502156971023-7698-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.132.131
--webserver_port=0
--tserver_master_addrs=127.7.132.190:35737
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.131
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502156971023-7698-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:37:36.276726 20118 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:37:36.278612 20118 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:37:36.293421 20126 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:37:36.293509 20125 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:37:36.293533 20128 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:37:36.294142 20118 server_base.cc:1034] running on GCE node
I20260812 02:37:36.294818 20118 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:37:36.295789 20118 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:37:36.297006 20118 hybrid_clock.cc:648] HybridClock initialized: now 1786502256296928 us; error 90 us; skew 500 ppm
I20260812 02:37:36.297356 20118 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 02:37:36.299810 20118 webserver.cc:460] Webserver started at http://127.7.132.131:43231/ using document root <none> and password file <none>
I20260812 02:37:36.300566 20118 fs_manager.cc:359] Metadata directory not provided
I20260812 02:37:36.300696 20118 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:37:36.301064 20118 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:37:36.304169 20118 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502156971023-7698-0/minicluster-data/ts-2/data/instance:
uuid: "ce87a6dfdcb6439895eea9ae06f87486"
format_stamp: "Formatted at 2026-08-12 02:37:36 on dist-test-slave-mn4r"
server_key: "cd0e4db262ab4c3ad0a6f2438a38d5b9"
server_key_iv: "65aa5fc00390cd08a18699469240e5b9"
server_key_version: "encryptionkey@0"
I20260812 02:37:36.305043 20118 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502156971023-7698-0/minicluster-data/ts-2/wal/instance:
uuid: "ce87a6dfdcb6439895eea9ae06f87486"
format_stamp: "Formatted at 2026-08-12 02:37:36 on dist-test-slave-mn4r"
server_key: "cd0e4db262ab4c3ad0a6f2438a38d5b9"
server_key_iv: "65aa5fc00390cd08a18699469240e5b9"
server_key_version: "encryptionkey@0"
I20260812 02:37:36.310887 20118 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260812 02:37:36.314749 20136 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:37:36.316211 20118 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260812 02:37:36.316377 20118 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502156971023-7698-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502156971023-7698-0/minicluster-data/ts-2/wal
uuid: "ce87a6dfdcb6439895eea9ae06f87486"
format_stamp: "Formatted at 2026-08-12 02:37:36 on dist-test-slave-mn4r"
server_key: "cd0e4db262ab4c3ad0a6f2438a38d5b9"
server_key_iv: "65aa5fc00390cd08a18699469240e5b9"
server_key_version: "encryptionkey@0"
I20260812 02:37:36.316614 20118 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502156971023-7698-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502156971023-7698-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502156971023-7698-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:37:36.322697 20118 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:37:36.323511 20118 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:37:36.323801 20118 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:37:36.324903 20118 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:37:36.326465 20118 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:37:36.326588 20118 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:37:36.326721 20118 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:37:36.326812 20118 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:37:36.362893 20118 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.131:34833
I20260812 02:37:36.362988 20270 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.131:34833 every 8 connection(s)
I20260812 02:37:36.364327 20118 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502156971023-7698-0/minicluster-data/ts-2/data/info.pb
I20260812 02:37:36.367443 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 20118
I20260812 02:37:36.367664 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786502156971023-7698-0/minicluster-data/ts-2/wal/instance
I20260812 02:37:36.368206 7698 external_mini_cluster.cc:1428] Setting key e724679848816610fa8cd869a012ff93
I20260812 02:37:36.373457 19954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:36.367112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:33365 (local address 127.7.132.190:35737)
0812 02:37:36.367384 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:36.367390 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:36.369184 (+ 1794us) server_negotiation.cc:407] Connection header received
0812 02:37:36.371095 (+ 1911us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:36.371101 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:36.371195 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:36.371307 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:37:36.372687 (+ 1380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:37:36.372693 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:37:36.372697 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:37:36.372716 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:37:36.372758 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:37:36.372769 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:37:36.372888 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0812 02:37:36.373085 (+ 197us) server_negotiation.cc:299] Negotiation successful
0812 02:37:36.373154 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
I20260812 02:37:36.374404 20275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:36.367832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:35737 (local address 127.7.132.131:33365)
0812 02:37:36.368997 (+ 1165us) negotiation.cc:107] Waiting for socket to connect
0812 02:37:36.369047 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 02:37:36.370827 (+ 1780us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:37:36.371577 (+ 750us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:37:36.371590 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:37:36.371720 (+ 130us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:37:36.372382 (+ 662us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 02:37:36.372384 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:37:36.372438 (+ 54us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 02:37:36.372524 (+ 86us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:37:36.372863 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:37:36.372873 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:37:36.372881 (+ 8us) client_negotiation.cc:769] Sending connection context
0812 02:37:36.373165 (+ 284us) client_negotiation.cc:240] Negotiation successful
0812 02:37:36.373411 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":539,"thread_start_us":345,"threads_started":1}
I20260812 02:37:36.375751 20271 heartbeater.cc:347] Connected to a master server at 127.7.132.190:35737
I20260812 02:37:36.376066 20271 heartbeater.cc:464] Registering TS with master...
I20260812 02:37:36.376686 20271 heartbeater.cc:511] Master 127.7.132.190:35737 requested a full tablet report, sending...
I20260812 02:37:36.378180 19716 ts_manager.cc:194] Registered new tserver with Master: ce87a6dfdcb6439895eea9ae06f87486 (127.7.132.131:34833)
I20260812 02:37:36.383992 7698 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260812 02:37:36.390308 19954 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:37:36.388606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35676 (local address 127.7.132.190:35737)
0812 02:37:36.388773 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:36.388778 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:36.389298 (+ 520us) server_negotiation.cc:407] Connection header received
0812 02:37:36.389508 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:36.389512 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:36.389567 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:36.389677 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:37:36.390038 (+ 361us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35676: BlockingRecv error: recv got EOF from 127.0.0.1:35676 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 02:37:36.391374 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 19792
I20260812 02:37:36.395184 20286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:36.392510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48482 (local address 127.7.132.129:36701)
0812 02:37:36.393054 (+ 544us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:36.393062 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:36.393089 (+ 27us) server_negotiation.cc:407] Connection header received
0812 02:37:36.393149 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:36.393155 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:36.393378 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:36.393511 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:37:36.394482 (+ 971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:37:36.394489 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:37:36.394501 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:37:36.394519 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:37:36.394559 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:37:36.394572 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:37:36.394677 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0812 02:37:36.394761 (+ 84us) server_negotiation.cc:299] Negotiation successful
0812 02:37:36.394874 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":383,"thread_start_us":262,"threads_started":1}
I20260812 02:37:36.396509 19946 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:37:36.478789 19952 heartbeater.cc:503] Master 127.7.132.190:35737 was elected leader, sending a full tablet report...
I20260812 02:37:36.480448 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 19792
W20260812 02:37:36.505053 19700 connection.cc:535] server connection from 127.7.132.129:55977 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 02:37:36.505314 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 19958
I20260812 02:37:36.509248 20288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:36.506357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57246 (local address 127.7.132.130:43717)
0812 02:37:36.506861 (+ 504us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:36.506876 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:36.506904 (+ 28us) server_negotiation.cc:407] Connection header received
0812 02:37:36.506969 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:36.506976 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:36.507207 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:36.507340 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:37:36.508380 (+ 1040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:37:36.508390 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:37:36.508412 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:37:36.508429 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:37:36.508488 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:37:36.508505 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:37:36.508652 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0812 02:37:36.508812 (+ 160us) server_negotiation.cc:299] Negotiation successful
0812 02:37:36.508903 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":361,"thread_start_us":247,"threads_started":1}
I20260812 02:37:36.510541 20108 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:37:36.592037 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 19958
I20260812 02:37:36.614337 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 20118
I20260812 02:37:36.618753 20290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:36.615384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41492 (local address 127.7.132.131:34833)
0812 02:37:36.616120 (+ 736us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:36.616127 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:36.616159 (+ 32us) server_negotiation.cc:407] Connection header received
0812 02:37:36.616230 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:36.616236 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:36.616451 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:36.616635 (+ 184us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:37:36.617888 (+ 1253us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:37:36.617894 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:37:36.617907 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:37:36.617924 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:37:36.617965 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:37:36.617979 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:37:36.618204 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0812 02:37:36.618322 (+ 118us) server_negotiation.cc:299] Negotiation successful
0812 02:37:36.618434 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":574,"thread_start_us":344,"threads_started":1}
I20260812 02:37:36.620050 20264 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:37:36.703226 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 20118
I20260812 02:37:36.726378 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 19678
I20260812 02:37:36.727699 19755 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:37:36.832749 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 19678
2026-08-12T02:37:36Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2487 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502156971023-7698-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 02:37:36 dist-test-slave-mn4r krb5kdc[20297](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 02:37:36 dist-test-slave-mn4r krb5kdc[20297](info): set up 2 sockets
Aug 12 02:37:36 dist-test-slave-mn4r krb5kdc[20297](info): commencing operation
krb5kdc: starting...
W20260812 02:37:38.122777 7698 mini_kdc.cc:121] Time spent starting KDC: real 1.243s user 0.003s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502156971023-7698-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502156971023-7698-0/krb5kdc/test-user.keytab.
Aug 12 02:37:38 dist-test-slave-mn4r krb5kdc[20297](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502258, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T02:37:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T02:37:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.132.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
I20260812 02:37:38.430790 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502156971023-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502156971023-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502156971023-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:39625
--webserver_interface=127.7.132.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:46361
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.132.190:39625
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:37:38.780567 20312 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:37:38.780946 20312 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:37:38.781122 20312 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:37:38.788501 20312 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:37:38.788679 20312 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:37:38.788795 20312 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:37:38.788904 20312 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:37:38.788993 20312 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:37:38.799096 20312 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:46361
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502156971023-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502156971023-7698-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.132.190:39625
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:39625
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.132.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502156971023-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:37:38.800621 20312 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:37:38.802063 20312 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:37:38.814783 20320 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:37:38.814808 20319 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:37:38.814894 20323 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:37:38.815610 20312 server_base.cc:1034] running on GCE node
I20260812 02:37:38.816437 20312 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:37:38.818049 20312 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:37:38.819346 20312 hybrid_clock.cc:648] HybridClock initialized: now 1786502258819257 us; error 78 us; skew 500 ppm
I20260812 02:37:38.819725 20312 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:37:38 dist-test-slave-mn4r krb5kdc[20297](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502258, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:37:38.823313 20312 init.cc:377] Logged in from keytab as kudu/127.7.132.190@KRBTEST.COM (short username kudu)
I20260812 02:37:38.824960 20312 webserver.cc:460] Webserver started at http://127.7.132.190:46743/ using document root <none> and password file <none>
I20260812 02:37:38.825752 20312 fs_manager.cc:359] Metadata directory not provided
I20260812 02:37:38.825906 20312 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:37:38.826328 20312 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:37:38.829769 20312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502156971023-7698-0/minicluster-data/master-0/data/instance:
uuid: "8a7a4dc8567d46848b01831d7bd6ac4d"
format_stamp: "Formatted at 2026-08-12 02:37:38 on dist-test-slave-mn4r"
server_key: "2bd05c5d74f754733e891e47299c4a3e"
server_key_iv: "9f2e712d00a1b16b70e6614f1fb430e3"
server_key_version: "encryptionkey@0"
I20260812 02:37:38.830826 20312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502156971023-7698-0/minicluster-data/master-0/wal/instance:
uuid: "8a7a4dc8567d46848b01831d7bd6ac4d"
format_stamp: "Formatted at 2026-08-12 02:37:38 on dist-test-slave-mn4r"
server_key: "2bd05c5d74f754733e891e47299c4a3e"
server_key_iv: "9f2e712d00a1b16b70e6614f1fb430e3"
server_key_version: "encryptionkey@0"
I20260812 02:37:38.836514 20312 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 02:37:38.840292 20330 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:37:38.841768 20312 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 02:37:38.841995 20312 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502156971023-7698-0/minicluster-data/master-0/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502156971023-7698-0/minicluster-data/master-0/wal
uuid: "8a7a4dc8567d46848b01831d7bd6ac4d"
format_stamp: "Formatted at 2026-08-12 02:37:38 on dist-test-slave-mn4r"
server_key: "2bd05c5d74f754733e891e47299c4a3e"
server_key_iv: "9f2e712d00a1b16b70e6614f1fb430e3"
server_key_version: "encryptionkey@0"
I20260812 02:37:38.842275 20312 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502156971023-7698-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502156971023-7698-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502156971023-7698-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:37:38.875763 20312 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:37:38.881451 20312 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:37:38.881734 20312 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:37:38.902663 20312 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.190:39625
I20260812 02:37:38.902683 20391 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.190:39625 every 8 connection(s)
I20260812 02:37:38.904289 20312 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
I20260812 02:37:38.908267 20392 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:37:38.910995 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 20312
I20260812 02:37:38.911257 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502156971023-7698-0/minicluster-data/master-0/wal/instance
I20260812 02:37:38.912076 7698 external_mini_cluster.cc:1428] Setting key 01fa76775edd7e5914a3346d03b66014
I20260812 02:37:38.921855 20392 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a7a4dc8567d46848b01831d7bd6ac4d: Bootstrap starting.
I20260812 02:37:38.925997 20392 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8a7a4dc8567d46848b01831d7bd6ac4d: Neither blocks nor log segments found. Creating new log.
I20260812 02:37:38.927475 20392 log.cc:824] T 00000000000000000000000000000000 P 8a7a4dc8567d46848b01831d7bd6ac4d: Log is configured to *not* fsync() on all Append() calls
Aug 12 02:37:38 dist-test-slave-mn4r krb5kdc[20297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502258, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 02:37:38.931176 20392 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a7a4dc8567d46848b01831d7bd6ac4d: No bootstrap required, opened a new log
I20260812 02:37:38.939145 20395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:38.914470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58104 (local address 127.7.132.190:39625)
0812 02:37:38.915349 (+ 879us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:38.915362 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:38.915417 (+ 55us) server_negotiation.cc:407] Connection header received
0812 02:37:38.917162 (+ 1745us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:38.917189 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:38.917625 (+ 436us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:38.917990 (+ 365us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:37:38.920022 (+ 2032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:38.921398 (+ 1376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:38.923351 (+ 1953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:38.924080 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:38.924231 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:38.929653 (+ 5422us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:37:38.929662 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:37:38.929678 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:37:38.929717 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:37:38.933874 (+ 4157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:37:38.934764 (+ 890us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:37:38.934775 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:37:38.934789 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:37:38.935019 (+ 230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:37:38.935481 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:37:38.935487 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:37:38.935490 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:37:38.936271 (+ 781us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:37:38.936585 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0812 02:37:38.937072 (+ 487us) server_negotiation.cc:299] Negotiation successful
0812 02:37:38.937583 (+ 511us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":658,"thread_start_us":410,"threads_started":1}
I20260812 02:37:38.942328 20392 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8a7a4dc8567d46848b01831d7bd6ac4d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a7a4dc8567d46848b01831d7bd6ac4d" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 39625 } }
I20260812 02:37:38.942804 20392 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8a7a4dc8567d46848b01831d7bd6ac4d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:37:38.942932 20392 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8a7a4dc8567d46848b01831d7bd6ac4d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a7a4dc8567d46848b01831d7bd6ac4d, State: Initialized, Role: FOLLOWER
I20260812 02:37:38.943583 20392 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8a7a4dc8567d46848b01831d7bd6ac4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a7a4dc8567d46848b01831d7bd6ac4d" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 39625 } }
I20260812 02:37:38.943835 20392 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8a7a4dc8567d46848b01831d7bd6ac4d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 02:37:38.943995 20392 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8a7a4dc8567d46848b01831d7bd6ac4d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 02:37:38.944178 20392 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8a7a4dc8567d46848b01831d7bd6ac4d [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:37:38.946799 20392 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8a7a4dc8567d46848b01831d7bd6ac4d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a7a4dc8567d46848b01831d7bd6ac4d" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 39625 } }
I20260812 02:37:38.947335 20392 leader_election.cc:302] T 00000000000000000000000000000000 P 8a7a4dc8567d46848b01831d7bd6ac4d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8a7a4dc8567d46848b01831d7bd6ac4d; no voters:
I20260812 02:37:38.947983 20392 leader_election.cc:288] T 00000000000000000000000000000000 P 8a7a4dc8567d46848b01831d7bd6ac4d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 02:37:38.948148 20397 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8a7a4dc8567d46848b01831d7bd6ac4d [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:37:38.949154 20397 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8a7a4dc8567d46848b01831d7bd6ac4d [term 1 LEADER]: Becoming Leader. State: Replica: 8a7a4dc8567d46848b01831d7bd6ac4d, State: Running, Role: LEADER
I20260812 02:37:38.949783 20397 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8a7a4dc8567d46848b01831d7bd6ac4d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a7a4dc8567d46848b01831d7bd6ac4d" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 39625 } }
I20260812 02:37:38.950353 20392 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8a7a4dc8567d46848b01831d7bd6ac4d [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:37:38.953646 20398 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a7a4dc8567d46848b01831d7bd6ac4d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8a7a4dc8567d46848b01831d7bd6ac4d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a7a4dc8567d46848b01831d7bd6ac4d" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 39625 } } }
I20260812 02:37:38.954263 20400 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a7a4dc8567d46848b01831d7bd6ac4d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8a7a4dc8567d46848b01831d7bd6ac4d. Latest consensus state: current_term: 1 leader_uuid: "8a7a4dc8567d46848b01831d7bd6ac4d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a7a4dc8567d46848b01831d7bd6ac4d" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 39625 } } }
I20260812 02:37:38.954572 20400 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a7a4dc8567d46848b01831d7bd6ac4d [sys.catalog]: This master's current role is: LEADER
I20260812 02:37:38.954039 20398 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a7a4dc8567d46848b01831d7bd6ac4d [sys.catalog]: This master's current role is: LEADER
I20260812 02:37:38.957932 20410 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:37:38.962206 20410 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 02:37:38.970799 20410 catalog_manager.cc:1347] Generated new cluster ID: fe087f95e90740edaf5ec350e30cb10b
I20260812 02:37:38.970906 20410 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 02:37:38.981549 20410 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 02:37:38.982977 20410 catalog_manager.cc:1504] Loading token signing keys...
I20260812 02:37:38.997180 20410 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8a7a4dc8567d46848b01831d7bd6ac4d: Generated new TSK 0
I20260812 02:37:38.998059 20410 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.132.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.129.keytab.
Entry for principal kudu/127.7.132.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.129.keytab.
I20260812 02:37:39.069445 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502156971023-7698-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502156971023-7698-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502156971023-7698-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502156971023-7698-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.129:0
--local_ip_for_outbound_sockets=127.7.132.129
--webserver_interface=127.7.132.129
--webserver_port=0
--tserver_master_addrs=127.7.132.190:39625
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.129.keytab
--principal=kudu/127.7.132.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:46361
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:37:39.423687 20425 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:37:39.423995 20425 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:37:39.424206 20425 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:37:39.431926 20425 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:37:39.432046 20425 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:37:39.432215 20425 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.129
I20260812 02:37:39.442584 20425 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:46361
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502156971023-7698-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502156971023-7698-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.129.keytab
--principal=kudu/127.7.132.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502156971023-7698-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.132.129
--webserver_port=0
--tserver_master_addrs=127.7.132.190:39625
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.129
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502156971023-7698-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:37:39.443991 20425 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:37:39.445354 20425 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:37:39.459182 20433 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:37:39.459209 20432 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:37:39.459837 20435 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:37:39.461419 20425 server_base.cc:1034] running on GCE node
I20260812 02:37:39.462172 20425 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:37:39.463178 20425 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:37:39.464413 20425 hybrid_clock.cc:648] HybridClock initialized: now 1786502259464364 us; error 41 us; skew 500 ppm
I20260812 02:37:39.464757 20425 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:37:39 dist-test-slave-mn4r krb5kdc[20297](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502259, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:37:39.468159 20425 init.cc:377] Logged in from keytab as kudu/127.7.132.129@KRBTEST.COM (short username kudu)
I20260812 02:37:39.469709 20425 webserver.cc:460] Webserver started at http://127.7.132.129:46319/ using document root <none> and password file <none>
I20260812 02:37:39.470525 20425 fs_manager.cc:359] Metadata directory not provided
I20260812 02:37:39.470681 20425 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:37:39.471005 20425 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:37:39.474043 20425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502156971023-7698-0/minicluster-data/ts-0/data/instance:
uuid: "636bdd0196d7455883f3b75888d4a6ee"
format_stamp: "Formatted at 2026-08-12 02:37:39 on dist-test-slave-mn4r"
server_key: "e16ab386c231466d7b78c0b9dca439a0"
server_key_iv: "73bd261db2c1eabc334d65c99550a305"
server_key_version: "encryptionkey@0"
I20260812 02:37:39.474938 20425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502156971023-7698-0/minicluster-data/ts-0/wal/instance:
uuid: "636bdd0196d7455883f3b75888d4a6ee"
format_stamp: "Formatted at 2026-08-12 02:37:39 on dist-test-slave-mn4r"
server_key: "e16ab386c231466d7b78c0b9dca439a0"
server_key_iv: "73bd261db2c1eabc334d65c99550a305"
server_key_version: "encryptionkey@0"
I20260812 02:37:39.480335 20425 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260812 02:37:39.483997 20442 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:37:39.485399 20425 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 02:37:39.485595 20425 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502156971023-7698-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502156971023-7698-0/minicluster-data/ts-0/wal
uuid: "636bdd0196d7455883f3b75888d4a6ee"
format_stamp: "Formatted at 2026-08-12 02:37:39 on dist-test-slave-mn4r"
server_key: "e16ab386c231466d7b78c0b9dca439a0"
server_key_iv: "73bd261db2c1eabc334d65c99550a305"
server_key_version: "encryptionkey@0"
I20260812 02:37:39.485814 20425 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502156971023-7698-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502156971023-7698-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502156971023-7698-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:37:39.502290 20425 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:37:39.506587 20425 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:37:39.506903 20425 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:37:39.507922 20425 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:37:39.509395 20425 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:37:39.509503 20425 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:37:39.509660 20425 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:37:39.509734 20425 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:37:39.544551 20425 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.129:44803
I20260812 02:37:39.544600 20576 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.129:44803 every 8 connection(s)
I20260812 02:37:39.546000 20425 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502156971023-7698-0/minicluster-data/ts-0/data/info.pb
I20260812 02:37:39.549415 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 20425
I20260812 02:37:39.549672 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502156971023-7698-0/minicluster-data/ts-0/wal/instance
I20260812 02:37:39.550446 7698 external_mini_cluster.cc:1428] Setting key cb4099ace81b6c475152ea93f68e138a
Aug 12 02:37:39 dist-test-slave-mn4r krb5kdc[20297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502259, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.129@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 02:37:39.567252 20579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:39.548835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:50487 (local address 127.7.132.190:39625)
0812 02:37:39.549389 (+ 554us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:39.549397 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:39.550640 (+ 1243us) server_negotiation.cc:407] Connection header received
0812 02:37:39.552609 (+ 1969us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:39.552614 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:39.552699 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:39.552828 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:37:39.554599 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:39.555343 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:39.556912 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:39.557331 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:39.557474 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:39.561286 (+ 3812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:37:39.561291 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:37:39.561295 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:37:39.561329 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:37:39.564030 (+ 2701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:37:39.564798 (+ 768us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:37:39.564803 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:37:39.564805 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:37:39.564925 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:37:39.565389 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:37:39.565393 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:37:39.565395 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:37:39.565661 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:37:39.565826 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0812 02:37:39.566615 (+ 789us) server_negotiation.cc:299] Negotiation successful
0812 02:37:39.566859 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":384,"thread_start_us":250,"threads_started":1}
I20260812 02:37:39.568293 20581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:39.549445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:39625 (local address 127.7.132.129:50487)
0812 02:37:39.550368 (+ 923us) negotiation.cc:107] Waiting for socket to connect
0812 02:37:39.550417 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 02:37:39.552331 (+ 1914us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:37:39.553138 (+ 807us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:37:39.553147 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:37:39.553705 (+ 558us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:37:39.554424 (+ 719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:39.554442 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:39.555514 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:39.555521 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:39.556753 (+ 1232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:39.556765 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:39.557581 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:39.557589 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:39.557846 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:39.558601 (+ 755us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:37:39.558624 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:37:39.561050 (+ 2426us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:37:39.564196 (+ 3146us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:37:39.564205 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:37:39.564222 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:37:39.564656 (+ 434us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:37:39.565042 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:37:39.565047 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:37:39.565050 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:37:39.565254 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:37:39.565797 (+ 543us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:37:39.565805 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:37:39.566226 (+ 421us) client_negotiation.cc:769] Sending connection context
0812 02:37:39.566496 (+ 270us) client_negotiation.cc:240] Negotiation successful
0812 02:37:39.567068 (+ 572us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":632,"thread_start_us":355,"threads_started":1}
I20260812 02:37:39.569988 20577 heartbeater.cc:347] Connected to a master server at 127.7.132.190:39625
I20260812 02:37:39.570348 20577 heartbeater.cc:464] Registering TS with master...
I20260812 02:37:39.571131 20577 heartbeater.cc:511] Master 127.7.132.190:39625 requested a full tablet report, sending...
I20260812 02:37:39.573608 20349 ts_manager.cc:194] Registered new tserver with Master: 636bdd0196d7455883f3b75888d4a6ee (127.7.132.129:44803)
I20260812 02:37:39.575950 20349 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.129@KRBTEST.COM'} at 127.7.132.129:50487
WARNING: no policy specified for kudu/127.7.132.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.130.keytab.
Entry for principal kudu/127.7.132.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.130.keytab.
I20260812 02:37:39.616928 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502156971023-7698-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502156971023-7698-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502156971023-7698-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502156971023-7698-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.130:0
--local_ip_for_outbound_sockets=127.7.132.130
--webserver_interface=127.7.132.130
--webserver_port=0
--tserver_master_addrs=127.7.132.190:39625
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.130.keytab
--principal=kudu/127.7.132.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:46361
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:37:39.963145 20584 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:37:39.963424 20584 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:37:39.963613 20584 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:37:39.970723 20584 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:37:39.970826 20584 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:37:39.970937 20584 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.130
I20260812 02:37:39.980819 20584 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:46361
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502156971023-7698-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502156971023-7698-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.130.keytab
--principal=kudu/127.7.132.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502156971023-7698-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.132.130
--webserver_port=0
--tserver_master_addrs=127.7.132.190:39625
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.130
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502156971023-7698-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:37:39.982157 20584 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:37:39.983593 20584 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:37:39.997438 20590 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:37:39.997679 20589 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:37:39.997362 20592 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:37:39.998795 20584 server_base.cc:1034] running on GCE node
I20260812 02:37:39.999457 20584 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:37:40.000458 20584 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:37:40.001705 20584 hybrid_clock.cc:648] HybridClock initialized: now 1786502260001644 us; error 45 us; skew 500 ppm
I20260812 02:37:40.002132 20584 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:37:40 dist-test-slave-mn4r krb5kdc[20297](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502260, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:37:40.005406 20584 init.cc:377] Logged in from keytab as kudu/127.7.132.130@KRBTEST.COM (short username kudu)
I20260812 02:37:40.007035 20584 webserver.cc:460] Webserver started at http://127.7.132.130:36801/ using document root <none> and password file <none>
I20260812 02:37:40.007800 20584 fs_manager.cc:359] Metadata directory not provided
I20260812 02:37:40.007951 20584 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:37:40.008270 20584 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:37:40.011632 20584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502156971023-7698-0/minicluster-data/ts-1/data/instance:
uuid: "19ecf8033bb54fffa782db078c8ff845"
format_stamp: "Formatted at 2026-08-12 02:37:40 on dist-test-slave-mn4r"
server_key: "a56c1fd821ed86e7083144c4a10a9782"
server_key_iv: "d13f344b1b5c258eb9b19177a353f887"
server_key_version: "encryptionkey@0"
I20260812 02:37:40.012482 20584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502156971023-7698-0/minicluster-data/ts-1/wal/instance:
uuid: "19ecf8033bb54fffa782db078c8ff845"
format_stamp: "Formatted at 2026-08-12 02:37:40 on dist-test-slave-mn4r"
server_key: "a56c1fd821ed86e7083144c4a10a9782"
server_key_iv: "d13f344b1b5c258eb9b19177a353f887"
server_key_version: "encryptionkey@0"
I20260812 02:37:40.017901 20584 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260812 02:37:40.021517 20601 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:37:40.022964 20584 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 02:37:40.023152 20584 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502156971023-7698-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502156971023-7698-0/minicluster-data/ts-1/wal
uuid: "19ecf8033bb54fffa782db078c8ff845"
format_stamp: "Formatted at 2026-08-12 02:37:40 on dist-test-slave-mn4r"
server_key: "a56c1fd821ed86e7083144c4a10a9782"
server_key_iv: "d13f344b1b5c258eb9b19177a353f887"
server_key_version: "encryptionkey@0"
I20260812 02:37:40.023360 20584 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502156971023-7698-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502156971023-7698-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502156971023-7698-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:37:40.038492 20584 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:37:40.042613 20584 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:37:40.042929 20584 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:37:40.043965 20584 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:37:40.045380 20584 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:37:40.045495 20584 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:37:40.045648 20584 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:37:40.045732 20584 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:37:40.080922 20584 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.130:46065
I20260812 02:37:40.080969 20732 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.130:46065 every 8 connection(s)
I20260812 02:37:40.082427 20584 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502156971023-7698-0/minicluster-data/ts-1/data/info.pb
I20260812 02:37:40.086032 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 20584
I20260812 02:37:40.086319 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502156971023-7698-0/minicluster-data/ts-1/wal/instance
I20260812 02:37:40.086928 7698 external_mini_cluster.cc:1428] Setting key 8f4635f20bc7accd221b6eee8b20bda8
Aug 12 02:37:40 dist-test-slave-mn4r krb5kdc[20297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502260, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.130@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 02:37:40.107240 20736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:40.085123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:34583 (local address 127.7.132.190:39625)
0812 02:37:40.085628 (+ 505us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:40.085635 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:40.086885 (+ 1250us) server_negotiation.cc:407] Connection header received
0812 02:37:40.089175 (+ 2290us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:40.089179 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:40.089243 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:40.089335 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:37:40.091609 (+ 2274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:40.092339 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:40.094876 (+ 2537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:40.095308 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:40.095409 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:40.100130 (+ 4721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:37:40.100135 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:37:40.100139 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:37:40.100174 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:37:40.103630 (+ 3456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:37:40.104436 (+ 806us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:37:40.104445 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:37:40.104452 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:37:40.104645 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:37:40.105074 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:37:40.105082 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:37:40.105088 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:37:40.105506 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:37:40.105672 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0812 02:37:40.106465 (+ 793us) server_negotiation.cc:299] Negotiation successful
0812 02:37:40.106747 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":360,"thread_start_us":234,"threads_started":1}
I20260812 02:37:40.107944 20737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:40.085725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:39625 (local address 127.7.132.130:34583)
0812 02:37:40.086661 (+ 936us) negotiation.cc:107] Waiting for socket to connect
0812 02:37:40.086729 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0812 02:37:40.088869 (+ 2140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:37:40.089641 (+ 772us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:37:40.089654 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:37:40.090491 (+ 837us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:37:40.091394 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:40.091425 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:40.092503 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:40.092511 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:40.094705 (+ 2194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:40.094723 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:40.095555 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:40.095566 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:40.095929 (+ 363us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:40.096697 (+ 768us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:37:40.096733 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:37:40.099834 (+ 3101us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:37:40.103804 (+ 3970us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:37:40.103811 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:37:40.103825 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:37:40.104283 (+ 458us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:37:40.104756 (+ 473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:37:40.104759 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:37:40.104761 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:37:40.104953 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:37:40.105630 (+ 677us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:37:40.105636 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:37:40.105977 (+ 341us) client_negotiation.cc:769] Sending connection context
0812 02:37:40.106297 (+ 320us) client_negotiation.cc:240] Negotiation successful
0812 02:37:40.106749 (+ 452us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":597,"thread_start_us":353,"threads_started":1}
I20260812 02:37:40.109560 20734 heartbeater.cc:347] Connected to a master server at 127.7.132.190:39625
I20260812 02:37:40.109916 20734 heartbeater.cc:464] Registering TS with master...
I20260812 02:37:40.110918 20734 heartbeater.cc:511] Master 127.7.132.190:39625 requested a full tablet report, sending...
I20260812 02:37:40.112648 20349 ts_manager.cc:194] Registered new tserver with Master: 19ecf8033bb54fffa782db078c8ff845 (127.7.132.130:46065)
I20260812 02:37:40.113500 20349 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.130@KRBTEST.COM'} at 127.7.132.130:34583
WARNING: no policy specified for kudu/127.7.132.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.131.keytab.
Entry for principal kudu/127.7.132.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.131.keytab.
I20260812 02:37:40.157491 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502156971023-7698-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502156971023-7698-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502156971023-7698-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502156971023-7698-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.131:0
--local_ip_for_outbound_sockets=127.7.132.131
--webserver_interface=127.7.132.131
--webserver_port=0
--tserver_master_addrs=127.7.132.190:39625
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.131.keytab
--principal=kudu/127.7.132.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:46361
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:37:40.506763 20741 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:37:40.507112 20741 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:37:40.507296 20741 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:37:40.514595 20741 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:37:40.514760 20741 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:37:40.514963 20741 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.131
I20260812 02:37:40.525182 20741 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:46361
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502156971023-7698-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502156971023-7698-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.131.keytab
--principal=kudu/127.7.132.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502156971023-7698-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.132.131
--webserver_port=0
--tserver_master_addrs=127.7.132.190:39625
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.131
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502156971023-7698-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:37:40.526610 20741 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:37:40.528283 20741 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:37:40.541913 20748 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:37:40.542025 20747 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:37:40.542066 20750 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:37:40.542761 20741 server_base.cc:1034] running on GCE node
I20260812 02:37:40.543427 20741 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:37:40.544646 20741 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:37:40.545962 20741 hybrid_clock.cc:648] HybridClock initialized: now 1786502260545854 us; error 55 us; skew 500 ppm
I20260812 02:37:40.546339 20741 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:37:40 dist-test-slave-mn4r krb5kdc[20297](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502260, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:37:40.549926 20741 init.cc:377] Logged in from keytab as kudu/127.7.132.131@KRBTEST.COM (short username kudu)
I20260812 02:37:40.551645 20741 webserver.cc:460] Webserver started at http://127.7.132.131:34171/ using document root <none> and password file <none>
I20260812 02:37:40.552368 20741 fs_manager.cc:359] Metadata directory not provided
I20260812 02:37:40.552522 20741 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:37:40.552842 20741 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:37:40.555989 20741 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502156971023-7698-0/minicluster-data/ts-2/data/instance:
uuid: "ba82dc6ba81f4b9d9037c2f95ed3bf02"
format_stamp: "Formatted at 2026-08-12 02:37:40 on dist-test-slave-mn4r"
server_key: "e7e5be8c2c50befd48e0c5f1f5bb2e3c"
server_key_iv: "3160ae7f9df99f8f2465bc51dbd78ff8"
server_key_version: "encryptionkey@0"
I20260812 02:37:40.556838 20741 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502156971023-7698-0/minicluster-data/ts-2/wal/instance:
uuid: "ba82dc6ba81f4b9d9037c2f95ed3bf02"
format_stamp: "Formatted at 2026-08-12 02:37:40 on dist-test-slave-mn4r"
server_key: "e7e5be8c2c50befd48e0c5f1f5bb2e3c"
server_key_iv: "3160ae7f9df99f8f2465bc51dbd78ff8"
server_key_version: "encryptionkey@0"
I20260812 02:37:40.562052 20741 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260812 02:37:40.565587 20757 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:37:40.567042 20741 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260812 02:37:40.567236 20741 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502156971023-7698-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502156971023-7698-0/minicluster-data/ts-2/wal
uuid: "ba82dc6ba81f4b9d9037c2f95ed3bf02"
format_stamp: "Formatted at 2026-08-12 02:37:40 on dist-test-slave-mn4r"
server_key: "e7e5be8c2c50befd48e0c5f1f5bb2e3c"
server_key_iv: "3160ae7f9df99f8f2465bc51dbd78ff8"
server_key_version: "encryptionkey@0"
I20260812 02:37:40.567438 20741 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502156971023-7698-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502156971023-7698-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502156971023-7698-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:37:40.579241 20577 heartbeater.cc:503] Master 127.7.132.190:39625 was elected leader, sending a full tablet report...
I20260812 02:37:40.585665 20741 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:37:40.589702 20741 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:37:40.590025 20741 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:37:40.591081 20741 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:37:40.592514 20741 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:37:40.592630 20741 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:37:40.592783 20741 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:37:40.592856 20741 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:37:40.626914 20741 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.131:34397
I20260812 02:37:40.627017 20892 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.131:34397 every 8 connection(s)
I20260812 02:37:40.628360 20741 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502156971023-7698-0/minicluster-data/ts-2/data/info.pb
I20260812 02:37:40.635943 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 20741
I20260812 02:37:40.636171 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502156971023-7698-0/minicluster-data/ts-2/wal/instance
I20260812 02:37:40.636713 7698 external_mini_cluster.cc:1428] Setting key cdcf94a6067a94d762caefdbdf910416
Aug 12 02:37:40 dist-test-slave-mn4r krb5kdc[20297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502260, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.131@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 02:37:40.649660 20895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:40.630988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:43285 (local address 127.7.132.190:39625)
0812 02:37:40.631414 (+ 426us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:40.631419 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:40.632417 (+ 998us) server_negotiation.cc:407] Connection header received
0812 02:37:40.634293 (+ 1876us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:40.634298 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:40.634377 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:40.634476 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:37:40.635943 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:40.637162 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:40.639314 (+ 2152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:40.639790 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:40.639907 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:40.643588 (+ 3681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:37:40.643594 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:37:40.643597 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:37:40.643633 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:37:40.646341 (+ 2708us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:37:40.647223 (+ 882us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:37:40.647229 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:37:40.647231 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:37:40.647361 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:37:40.647846 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:37:40.647850 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:37:40.647852 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:37:40.648135 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:37:40.648312 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0812 02:37:40.649035 (+ 723us) server_negotiation.cc:299] Negotiation successful
0812 02:37:40.649274 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":313,"thread_start_us":205,"threads_started":1}
I20260812 02:37:40.650826 20896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:40.631567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:39625 (local address 127.7.132.131:43285)
0812 02:37:40.632260 (+ 693us) negotiation.cc:107] Waiting for socket to connect
0812 02:37:40.632298 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 02:37:40.633973 (+ 1675us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:37:40.634689 (+ 716us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:37:40.634696 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:37:40.635204 (+ 508us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:37:40.635758 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:40.635775 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:40.637377 (+ 1602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:40.637384 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:40.639171 (+ 1787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:40.639186 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:40.640061 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:40.640070 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:40.640287 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:40.641010 (+ 723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:37:40.641043 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:37:40.643340 (+ 2297us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:37:40.646522 (+ 3182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:37:40.646538 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:37:40.646552 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:37:40.647083 (+ 531us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:37:40.647481 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:37:40.647485 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:37:40.647489 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:37:40.647706 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:37:40.648281 (+ 575us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:37:40.648289 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:37:40.648645 (+ 356us) client_negotiation.cc:769] Sending connection context
0812 02:37:40.648921 (+ 276us) client_negotiation.cc:240] Negotiation successful
0812 02:37:40.649468 (+ 547us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":260,"threads_started":1}
I20260812 02:37:40.652412 20893 heartbeater.cc:347] Connected to a master server at 127.7.132.190:39625
I20260812 02:37:40.652772 20893 heartbeater.cc:464] Registering TS with master...
I20260812 02:37:40.653597 20893 heartbeater.cc:511] Master 127.7.132.190:39625 requested a full tablet report, sending...
I20260812 02:37:40.655092 20349 ts_manager.cc:194] Registered new tserver with Master: ba82dc6ba81f4b9d9037c2f95ed3bf02 (127.7.132.131:34397)
I20260812 02:37:40.655913 20349 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.131@KRBTEST.COM'} at 127.7.132.131:43285
I20260812 02:37:40.664853 7698 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 02:37:40.682569 20895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:40.669398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58120 (local address 127.7.132.190:39625)
0812 02:37:40.669645 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:40.669654 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:40.670053 (+ 399us) server_negotiation.cc:407] Connection header received
0812 02:37:40.670263 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:40.670268 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:40.670350 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:40.670435 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:37:40.671989 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:40.672748 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:40.674014 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:40.674511 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:40.674662 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:40.676982 (+ 2320us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:37:40.676988 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:37:40.676993 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:37:40.677043 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:37:40.679784 (+ 2741us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:37:40.680467 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:37:40.680474 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:37:40.680478 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:37:40.680620 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:37:40.681009 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:37:40.681014 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:37:40.681019 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:37:40.681355 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:37:40.681633 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0812 02:37:40.681993 (+ 360us) server_negotiation.cc:299] Negotiation successful
0812 02:37:40.682249 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260812 02:37:40.686806 20349 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58120:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 02:37:40.693943 20349 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 02:37:40.718007 20906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:40.706290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.131:34397 (local address 127.0.0.1:56240)
0812 02:37:40.708308 (+ 2018us) negotiation.cc:107] Waiting for socket to connect
0812 02:37:40.708325 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 02:37:40.708452 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:37:40.709195 (+ 743us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:37:40.709203 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:37:40.709234 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:37:40.709688 (+ 454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:40.709700 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:40.712117 (+ 2417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:40.712123 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:40.714927 (+ 2804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:40.714942 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:40.716826 (+ 1884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:40.716835 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:40.717057 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:40.717111 (+ 54us) client_negotiation.cc:769] Sending connection context
0812 02:37:40.717218 (+ 107us) client_negotiation.cc:240] Negotiation successful
0812 02:37:40.717431 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1905,"mutex_wait_us":88,"thread_start_us":925,"threads_started":1}
I20260812 02:37:40.718503 20909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:40.706613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56240 (local address 127.7.132.131:34397)
0812 02:37:40.708318 (+ 1705us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:40.708329 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:40.708396 (+ 67us) server_negotiation.cc:407] Connection header received
0812 02:37:40.708652 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:40.708664 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:40.708933 (+ 269us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:40.709908 (+ 975us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:37:40.710221 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:40.711938 (+ 1717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:40.715122 (+ 3184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:40.716590 (+ 1468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:40.717655 (+ 1065us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:40.717907 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0812 02:37:40.717991 (+ 84us) server_negotiation.cc:299] Negotiation successful
0812 02:37:40.718178 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1518,"thread_start_us":1318,"threads_started":1}
I20260812 02:37:40.718690 20907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:40.706278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54176 (local address 127.7.132.129:44803)
0812 02:37:40.709091 (+ 2813us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:40.709102 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:40.709130 (+ 28us) server_negotiation.cc:407] Connection header received
0812 02:37:40.709218 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:40.709224 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:40.709443 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:40.710363 (+ 920us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:37:40.710617 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:40.712096 (+ 1479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:40.715441 (+ 3345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:40.716887 (+ 1446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:40.717016 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:40.717259 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0812 02:37:40.718163 (+ 904us) server_negotiation.cc:299] Negotiation successful
0812 02:37:40.718347 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2631,"thread_start_us":2467,"threads_started":1}
I20260812 02:37:40.719120 20908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:40.706067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:44803 (local address 127.0.0.1:54176)
0812 02:37:40.707582 (+ 1515us) negotiation.cc:107] Waiting for socket to connect
0812 02:37:40.707634 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0812 02:37:40.707793 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:37:40.709688 (+ 1895us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:37:40.709692 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:37:40.709718 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:37:40.710162 (+ 444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:40.710178 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:40.712248 (+ 2070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:40.712253 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:40.715269 (+ 3016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:40.715283 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:40.717407 (+ 2124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:40.717413 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:40.717633 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:40.717669 (+ 36us) client_negotiation.cc:769] Sending connection context
0812 02:37:40.717764 (+ 95us) client_negotiation.cc:240] Negotiation successful
0812 02:37:40.717906 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1245,"mutex_wait_us":69,"thread_start_us":2109,"threads_started":1}
I20260812 02:37:40.722856 20910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:40.708657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.130:46065 (local address 127.0.0.1:34540)
0812 02:37:40.713474 (+ 4817us) negotiation.cc:107] Waiting for socket to connect
0812 02:37:40.713493 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 02:37:40.713617 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:37:40.715373 (+ 1756us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:37:40.715377 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:37:40.715402 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:37:40.715774 (+ 372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:40.715783 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:40.717502 (+ 1719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:40.717510 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:40.720328 (+ 2818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:40.720341 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:40.722059 (+ 1718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:40.722065 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:40.722248 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:40.722288 (+ 40us) client_negotiation.cc:769] Sending connection context
0812 02:37:40.722403 (+ 115us) client_negotiation.cc:240] Negotiation successful
0812 02:37:40.722524 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4683,"thread_start_us":4526,"threads_started":1}
I20260812 02:37:40.722896 20911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:40.709191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34540 (local address 127.7.132.130:46065)
0812 02:37:40.714907 (+ 5716us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:40.714913 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:40.714937 (+ 24us) server_negotiation.cc:407] Connection header received
0812 02:37:40.715008 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:40.715013 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:40.715180 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:40.715384 (+ 204us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:37:40.715961 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:40.717326 (+ 1365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:40.720488 (+ 3162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:40.721829 (+ 1341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:40.722011 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:40.722247 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0812 02:37:40.722448 (+ 201us) server_negotiation.cc:299] Negotiation successful
0812 02:37:40.722604 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5556,"thread_start_us":5449,"threads_started":1}
I20260812 02:37:40.722697 20814 tablet_service.cc:1467] Processing CreateTablet for tablet 6f8cc98d5a2c49bdb9489d4fd9ca5596 (DEFAULT_TABLE table=test-table [id=f2c4f724bf5b426e8a665e31fbcb571a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:37:40.722697 20501 tablet_service.cc:1467] Processing CreateTablet for tablet 6f8cc98d5a2c49bdb9489d4fd9ca5596 (DEFAULT_TABLE table=test-table [id=f2c4f724bf5b426e8a665e31fbcb571a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:37:40.724272 20501 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f8cc98d5a2c49bdb9489d4fd9ca5596. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:37:40.724291 20814 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f8cc98d5a2c49bdb9489d4fd9ca5596. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:37:40.726637 20661 tablet_service.cc:1467] Processing CreateTablet for tablet 6f8cc98d5a2c49bdb9489d4fd9ca5596 (DEFAULT_TABLE table=test-table [id=f2c4f724bf5b426e8a665e31fbcb571a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:37:40.728025 20661 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f8cc98d5a2c49bdb9489d4fd9ca5596. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:37:40.739732 20912 tablet_bootstrap.cc:492] T 6f8cc98d5a2c49bdb9489d4fd9ca5596 P 636bdd0196d7455883f3b75888d4a6ee: Bootstrap starting.
I20260812 02:37:40.740717 20913 tablet_bootstrap.cc:492] T 6f8cc98d5a2c49bdb9489d4fd9ca5596 P ba82dc6ba81f4b9d9037c2f95ed3bf02: Bootstrap starting.
I20260812 02:37:40.742355 20914 tablet_bootstrap.cc:492] T 6f8cc98d5a2c49bdb9489d4fd9ca5596 P 19ecf8033bb54fffa782db078c8ff845: Bootstrap starting.
I20260812 02:37:40.744522 20913 tablet_bootstrap.cc:654] T 6f8cc98d5a2c49bdb9489d4fd9ca5596 P ba82dc6ba81f4b9d9037c2f95ed3bf02: Neither blocks nor log segments found. Creating new log.
I20260812 02:37:40.745210 20912 tablet_bootstrap.cc:654] T 6f8cc98d5a2c49bdb9489d4fd9ca5596 P 636bdd0196d7455883f3b75888d4a6ee: Neither blocks nor log segments found. Creating new log.
I20260812 02:37:40.745700 20913 log.cc:824] T 6f8cc98d5a2c49bdb9489d4fd9ca5596 P ba82dc6ba81f4b9d9037c2f95ed3bf02: Log is configured to *not* fsync() on all Append() calls
I20260812 02:37:40.745914 20914 tablet_bootstrap.cc:654] T 6f8cc98d5a2c49bdb9489d4fd9ca5596 P 19ecf8033bb54fffa782db078c8ff845: Neither blocks nor log segments found. Creating new log.
I20260812 02:37:40.746546 20912 log.cc:824] T 6f8cc98d5a2c49bdb9489d4fd9ca5596 P 636bdd0196d7455883f3b75888d4a6ee: Log is configured to *not* fsync() on all Append() calls
I20260812 02:37:40.747265 20914 log.cc:824] T 6f8cc98d5a2c49bdb9489d4fd9ca5596 P 19ecf8033bb54fffa782db078c8ff845: Log is configured to *not* fsync() on all Append() calls
I20260812 02:37:40.749078 20913 tablet_bootstrap.cc:492] T 6f8cc98d5a2c49bdb9489d4fd9ca5596 P ba82dc6ba81f4b9d9037c2f95ed3bf02: No bootstrap required, opened a new log
I20260812 02:37:40.749382 20913 ts_tablet_manager.cc:1397] T 6f8cc98d5a2c49bdb9489d4fd9ca5596 P ba82dc6ba81f4b9d9037c2f95ed3bf02: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260812 02:37:40.750545 20914 tablet_bootstrap.cc:492] T 6f8cc98d5a2c49bdb9489d4fd9ca5596 P 19ecf8033bb54fffa782db078c8ff845: No bootstrap required, opened a new log
I20260812 02:37:40.750820 20914 ts_tablet_manager.cc:1397] T 6f8cc98d5a2c49bdb9489d4fd9ca5596 P 19ecf8033bb54fffa782db078c8ff845: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260812 02:37:40.751487 20912 tablet_bootstrap.cc:492] T 6f8cc98d5a2c49bdb9489d4fd9ca5596 P 636bdd0196d7455883f3b75888d4a6ee: No bootstrap required, opened a new log
I20260812 02:37:40.751816 20912 ts_tablet_manager.cc:1397] T 6f8cc98d5a2c49bdb9489d4fd9ca5596 P 636bdd0196d7455883f3b75888d4a6ee: Time spent bootstrapping tablet: real 0.012s user 0.009s sys 0.000s
I20260812 02:37:40.757400 20913 raft_consensus.cc:348] T 6f8cc98d5a2c49bdb9489d4fd9ca5596 P ba82dc6ba81f4b9d9037c2f95ed3bf02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "636bdd0196d7455883f3b75888d4a6ee" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 44803 } } peers { permanent_uuid: "ba82dc6ba81f4b9d9037c2f95ed3bf02" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 34397 } } peers { permanent_uuid: "19ecf8033bb54fffa782db078c8ff845" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 46065 } }
I20260812 02:37:40.758011 20913 raft_consensus.cc:374] T 6f8cc98d5a2c49bdb9489d4fd9ca5596 P ba82dc6ba81f4b9d9037c2f95ed3bf02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:37:40.758224 20913 raft_consensus.cc:729] T 6f8cc98d5a2c49bdb9489d4fd9ca5596 P ba82dc6ba81f4b9d9037c2f95ed3bf02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba82dc6ba81f4b9d9037c2f95ed3bf02, State: Initialized, Role: FOLLOWER
I20260812 02:37:40.759043 20913 consensus_queue.cc:260] T 6f8cc98d5a2c49bdb9489d4fd9ca5596 P ba82dc6ba81f4b9d9037c2f95ed3bf02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "636bdd0196d7455883f3b75888d4a6ee" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 44803 } } peers { permanent_uuid: "ba82dc6ba81f4b9d9037c2f95ed3bf02" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 34397 } } peers { permanent_uuid: "19ecf8033bb54fffa782db078c8ff845" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 46065 } }
I20260812 02:37:40.760186 20893 heartbeater.cc:503] Master 127.7.132.190:39625 was elected leader, sending a full tablet report...
I20260812 02:37:40.760746 20913 ts_tablet_manager.cc:1428] T 6f8cc98d5a2c49bdb9489d4fd9ca5596 P ba82dc6ba81f4b9d9037c2f95ed3bf02: Time spent starting tablet: real 0.011s user 0.005s sys 0.004s
I20260812 02:37:40.761409 20914 raft_consensus.cc:348] T 6f8cc98d5a2c49bdb9489d4fd9ca5596 P 19ecf8033bb54fffa782db078c8ff845 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "636bdd0196d7455883f3b75888d4a6ee" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 44803 } } peers { permanent_uuid: "ba82dc6ba81f4b9d9037c2f95ed3bf02" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 34397 } } peers { permanent_uuid: "19ecf8033bb54fffa782db078c8ff845" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 46065 } }
I20260812 02:37:40.761905 20914 raft_consensus.cc:374] T 6f8cc98d5a2c49bdb9489d4fd9ca5596 P 19ecf8033bb54fffa782db078c8ff845 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:37:40.762048 20914 raft_consensus.cc:729] T 6f8cc98d5a2c49bdb9489d4fd9ca5596 P 19ecf8033bb54fffa782db078c8ff845 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19ecf8033bb54fffa782db078c8ff845, State: Initialized, Role: FOLLOWER
I20260812 02:37:40.762683 20914 consensus_queue.cc:260] T 6f8cc98d5a2c49bdb9489d4fd9ca5596 P 19ecf8033bb54fffa782db078c8ff845 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "636bdd0196d7455883f3b75888d4a6ee" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 44803 } } peers { permanent_uuid: "ba82dc6ba81f4b9d9037c2f95ed3bf02" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 34397 } } peers { permanent_uuid: "19ecf8033bb54fffa782db078c8ff845" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 46065 } }
I20260812 02:37:40.763294 20912 raft_consensus.cc:348] T 6f8cc98d5a2c49bdb9489d4fd9ca5596 P 636bdd0196d7455883f3b75888d4a6ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "636bdd0196d7455883f3b75888d4a6ee" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 44803 } } peers { permanent_uuid: "ba82dc6ba81f4b9d9037c2f95ed3bf02" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 34397 } } peers { permanent_uuid: "19ecf8033bb54fffa782db078c8ff845" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 46065 } }
I20260812 02:37:40.763782 20912 raft_consensus.cc:374] T 6f8cc98d5a2c49bdb9489d4fd9ca5596 P 636bdd0196d7455883f3b75888d4a6ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:37:40.763805 20734 heartbeater.cc:503] Master 127.7.132.190:39625 was elected leader, sending a full tablet report...
I20260812 02:37:40.763928 20912 raft_consensus.cc:729] T 6f8cc98d5a2c49bdb9489d4fd9ca5596 P 636bdd0196d7455883f3b75888d4a6ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 636bdd0196d7455883f3b75888d4a6ee, State: Initialized, Role: FOLLOWER
I20260812 02:37:40.764503 20914 ts_tablet_manager.cc:1428] T 6f8cc98d5a2c49bdb9489d4fd9ca5596 P 19ecf8033bb54fffa782db078c8ff845: Time spent starting tablet: real 0.014s user 0.009s sys 0.004s
I20260812 02:37:40.764597 20912 consensus_queue.cc:260] T 6f8cc98d5a2c49bdb9489d4fd9ca5596 P 636bdd0196d7455883f3b75888d4a6ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "636bdd0196d7455883f3b75888d4a6ee" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 44803 } } peers { permanent_uuid: "ba82dc6ba81f4b9d9037c2f95ed3bf02" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 34397 } } peers { permanent_uuid: "19ecf8033bb54fffa782db078c8ff845" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 46065 } }
I20260812 02:37:40.766291 20912 ts_tablet_manager.cc:1428] T 6f8cc98d5a2c49bdb9489d4fd9ca5596 P 636bdd0196d7455883f3b75888d4a6ee: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
W20260812 02:37:40.798811 20578 tablet.cc:2369] T 6f8cc98d5a2c49bdb9489d4fd9ca5596 P 636bdd0196d7455883f3b75888d4a6ee: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 02:37:40.834510 20735 tablet.cc:2369] T 6f8cc98d5a2c49bdb9489d4fd9ca5596 P 19ecf8033bb54fffa782db078c8ff845: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 02:37:40.880115 20894 tablet.cc:2369] T 6f8cc98d5a2c49bdb9489d4fd9ca5596 P ba82dc6ba81f4b9d9037c2f95ed3bf02: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 02:37:40.920485 20919 raft_consensus.cc:482] T 6f8cc98d5a2c49bdb9489d4fd9ca5596 P 19ecf8033bb54fffa782db078c8ff845 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 02:37:40.920881 20919 raft_consensus.cc:504] T 6f8cc98d5a2c49bdb9489d4fd9ca5596 P 19ecf8033bb54fffa782db078c8ff845 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "636bdd0196d7455883f3b75888d4a6ee" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 44803 } } peers { permanent_uuid: "ba82dc6ba81f4b9d9037c2f95ed3bf02" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 34397 } } peers { permanent_uuid: "19ecf8033bb54fffa782db078c8ff845" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 46065 } }
I20260812 02:37:40.922873 20919 leader_election.cc:288] T 6f8cc98d5a2c49bdb9489d4fd9ca5596 P 19ecf8033bb54fffa782db078c8ff845 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 636bdd0196d7455883f3b75888d4a6ee (127.7.132.129:44803), ba82dc6ba81f4b9d9037c2f95ed3bf02 (127.7.132.131:34397)
I20260812 02:37:40.930302 20907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:40.923448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:46249 (local address 127.7.132.129:44803)
0812 02:37:40.923741 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:40.923750 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:40.924467 (+ 717us) server_negotiation.cc:407] Connection header received
0812 02:37:40.924718 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:40.924722 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:40.924784 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:40.924886 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:37:40.925471 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:40.926319 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:40.928425 (+ 2106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:40.929251 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:40.929455 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:40.929541 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0812 02:37:40.929863 (+ 322us) server_negotiation.cc:299] Negotiation successful
0812 02:37:40.929996 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
I20260812 02:37:40.930388 20909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:40.923629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:41433 (local address 127.7.132.131:34397)
0812 02:37:40.923895 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:40.923903 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:40.924634 (+ 731us) server_negotiation.cc:407] Connection header received
0812 02:37:40.924853 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:40.924857 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:40.924919 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:40.925028 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:37:40.925697 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:40.926582 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:40.928503 (+ 1921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:40.929302 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:40.929537 (+ 235us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:40.929627 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0812 02:37:40.929948 (+ 321us) server_negotiation.cc:299] Negotiation successful
0812 02:37:40.930120 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260812 02:37:40.930485 20921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:40.923186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:44803 (local address 127.7.132.130:46249)
0812 02:37:40.924328 (+ 1142us) negotiation.cc:107] Waiting for socket to connect
0812 02:37:40.924352 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0812 02:37:40.924556 (+ 204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:37:40.924903 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:37:40.924908 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:37:40.924933 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:37:40.925316 (+ 383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:40.925326 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:40.926445 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:40.926451 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:40.928264 (+ 1813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:40.928277 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:40.929510 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:40.929516 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:40.929695 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:40.929724 (+ 29us) client_negotiation.cc:769] Sending connection context
0812 02:37:40.929834 (+ 110us) client_negotiation.cc:240] Negotiation successful
0812 02:37:40.929980 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1009,"mutex_wait_us":1,"thread_start_us":792,"threads_started":1}
I20260812 02:37:40.930543 20922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:40.923350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.131:34397 (local address 127.7.132.130:41433)
0812 02:37:40.924505 (+ 1155us) negotiation.cc:107] Waiting for socket to connect
0812 02:37:40.924520 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 02:37:40.924685 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:37:40.925045 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:37:40.925049 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:37:40.925098 (+ 49us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:37:40.925513 (+ 415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:40.925522 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:40.926723 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:40.926728 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:40.928339 (+ 1611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:37:40.928352 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:37:40.929567 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:37:40.929573 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:37:40.929775 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:40.929798 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 02:37:40.929907 (+ 109us) client_negotiation.cc:240] Negotiation successful
0812 02:37:40.930051 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1040,"mutex_wait_us":1,"thread_start_us":794,"threads_started":1}
I20260812 02:37:40.931761 20841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f8cc98d5a2c49bdb9489d4fd9ca5596" candidate_uuid: "19ecf8033bb54fffa782db078c8ff845" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba82dc6ba81f4b9d9037c2f95ed3bf02" is_pre_election: true
I20260812 02:37:40.931761 20521 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f8cc98d5a2c49bdb9489d4fd9ca5596" candidate_uuid: "19ecf8033bb54fffa782db078c8ff845" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "636bdd0196d7455883f3b75888d4a6ee" is_pre_election: true
I20260812 02:37:40.932310 20841 raft_consensus.cc:2454] T 6f8cc98d5a2c49bdb9489d4fd9ca5596 P ba82dc6ba81f4b9d9037c2f95ed3bf02 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19ecf8033bb54fffa782db078c8ff845 in term 0.
I20260812 02:37:40.932304 20521 raft_consensus.cc:2454] T 6f8cc98d5a2c49bdb9489d4fd9ca5596 P 636bdd0196d7455883f3b75888d4a6ee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19ecf8033bb54fffa782db078c8ff845 in term 0.
I20260812 02:37:40.933056 20605 leader_election.cc:302] T 6f8cc98d5a2c49bdb9489d4fd9ca5596 P 19ecf8033bb54fffa782db078c8ff845 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19ecf8033bb54fffa782db078c8ff845, ba82dc6ba81f4b9d9037c2f95ed3bf02; no voters:
I20260812 02:37:40.933490 20919 raft_consensus.cc:2781] T 6f8cc98d5a2c49bdb9489d4fd9ca5596 P 19ecf8033bb54fffa782db078c8ff845 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 02:37:40.933656 20919 raft_consensus.cc:482] T 6f8cc98d5a2c49bdb9489d4fd9ca5596 P 19ecf8033bb54fffa782db078c8ff845 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 02:37:40.933772 20919 raft_consensus.cc:3037] T 6f8cc98d5a2c49bdb9489d4fd9ca5596 P 19ecf8033bb54fffa782db078c8ff845 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:37:40.936156 20919 raft_consensus.cc:504] T 6f8cc98d5a2c49bdb9489d4fd9ca5596 P 19ecf8033bb54fffa782db078c8ff845 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "636bdd0196d7455883f3b75888d4a6ee" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 44803 } } peers { permanent_uuid: "ba82dc6ba81f4b9d9037c2f95ed3bf02" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 34397 } } peers { permanent_uuid: "19ecf8033bb54fffa782db078c8ff845" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 46065 } }
I20260812 02:37:40.936812 20919 leader_election.cc:288] T 6f8cc98d5a2c49bdb9489d4fd9ca5596 P 19ecf8033bb54fffa782db078c8ff845 [CANDIDATE]: Term 1 election: Requested vote from peers 636bdd0196d7455883f3b75888d4a6ee (127.7.132.129:44803), ba82dc6ba81f4b9d9037c2f95ed3bf02 (127.7.132.131:34397)
I20260812 02:37:40.937305 20521 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f8cc98d5a2c49bdb9489d4fd9ca5596" candidate_uuid: "19ecf8033bb54fffa782db078c8ff845" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "636bdd0196d7455883f3b75888d4a6ee"
I20260812 02:37:40.937356 20841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f8cc98d5a2c49bdb9489d4fd9ca5596" candidate_uuid: "19ecf8033bb54fffa782db078c8ff845" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba82dc6ba81f4b9d9037c2f95ed3bf02"
I20260812 02:37:40.937522 20521 raft_consensus.cc:3037] T 6f8cc98d5a2c49bdb9489d4fd9ca5596 P 636bdd0196d7455883f3b75888d4a6ee [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:37:40.937564 20841 raft_consensus.cc:3037] T 6f8cc98d5a2c49bdb9489d4fd9ca5596 P ba82dc6ba81f4b9d9037c2f95ed3bf02 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:37:40.939877 20841 raft_consensus.cc:2454] T 6f8cc98d5a2c49bdb9489d4fd9ca5596 P ba82dc6ba81f4b9d9037c2f95ed3bf02 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19ecf8033bb54fffa782db078c8ff845 in term 1.
I20260812 02:37:40.939898 20521 raft_consensus.cc:2454] T 6f8cc98d5a2c49bdb9489d4fd9ca5596 P 636bdd0196d7455883f3b75888d4a6ee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19ecf8033bb54fffa782db078c8ff845 in term 1.
I20260812 02:37:40.940333 20605 leader_election.cc:302] T 6f8cc98d5a2c49bdb9489d4fd9ca5596 P 19ecf8033bb54fffa782db078c8ff845 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19ecf8033bb54fffa782db078c8ff845, ba82dc6ba81f4b9d9037c2f95ed3bf02; no voters:
I20260812 02:37:40.940694 20919 raft_consensus.cc:2781] T 6f8cc98d5a2c49bdb9489d4fd9ca5596 P 19ecf8033bb54fffa782db078c8ff845 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:37:40.941218 20919 raft_consensus.cc:686] T 6f8cc98d5a2c49bdb9489d4fd9ca5596 P 19ecf8033bb54fffa782db078c8ff845 [term 1 LEADER]: Becoming Leader. State: Replica: 19ecf8033bb54fffa782db078c8ff845, State: Running, Role: LEADER
I20260812 02:37:40.941798 20919 consensus_queue.cc:237] T 6f8cc98d5a2c49bdb9489d4fd9ca5596 P 19ecf8033bb54fffa782db078c8ff845 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "636bdd0196d7455883f3b75888d4a6ee" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 44803 } } peers { permanent_uuid: "ba82dc6ba81f4b9d9037c2f95ed3bf02" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 34397 } } peers { permanent_uuid: "19ecf8033bb54fffa782db078c8ff845" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 46065 } }
I20260812 02:37:40.947233 20348 catalog_manager.cc:5495] T 6f8cc98d5a2c49bdb9489d4fd9ca5596 P 19ecf8033bb54fffa782db078c8ff845 reported cstate change: term changed from 0 to 1, leader changed from <none> to 19ecf8033bb54fffa782db078c8ff845 (127.7.132.130). New cstate: current_term: 1 leader_uuid: "19ecf8033bb54fffa782db078c8ff845" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "636bdd0196d7455883f3b75888d4a6ee" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 44803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba82dc6ba81f4b9d9037c2f95ed3bf02" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 34397 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "19ecf8033bb54fffa782db078c8ff845" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 46065 } health_report { overall_health: HEALTHY } } }
I20260812 02:37:40.969365 20911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:40.964077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34546 (local address 127.7.132.130:46065)
0812 02:37:40.964264 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:40.964269 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:40.964287 (+ 18us) server_negotiation.cc:407] Connection header received
0812 02:37:40.964376 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:40.964379 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:40.964456 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:40.964592 (+ 136us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 02:37:40.964912 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:40.965628 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:40.967036 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:40.967557 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:40.967652 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:40.968227 (+ 575us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 02:37:40.968747 (+ 520us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 02:37:40.968883 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0812 02:37:40.968969 (+ 86us) server_negotiation.cc:299] Negotiation successful
0812 02:37:40.969037 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 02:37:40.977960 20841 raft_consensus.cc:1261] T 6f8cc98d5a2c49bdb9489d4fd9ca5596 P ba82dc6ba81f4b9d9037c2f95ed3bf02 [term 1 FOLLOWER]: Refusing update from remote peer 19ecf8033bb54fffa782db078c8ff845: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:37:40.977959 20521 raft_consensus.cc:1261] T 6f8cc98d5a2c49bdb9489d4fd9ca5596 P 636bdd0196d7455883f3b75888d4a6ee [term 1 FOLLOWER]: Refusing update from remote peer 19ecf8033bb54fffa782db078c8ff845: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:37:40.979157 20923 consensus_queue.cc:1035] T 6f8cc98d5a2c49bdb9489d4fd9ca5596 P 19ecf8033bb54fffa782db078c8ff845 [LEADER]: Connected to new peer: Peer: permanent_uuid: "636bdd0196d7455883f3b75888d4a6ee" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 44803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:37:40.979532 20919 consensus_queue.cc:1035] T 6f8cc98d5a2c49bdb9489d4fd9ca5596 P 19ecf8033bb54fffa782db078c8ff845 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba82dc6ba81f4b9d9037c2f95ed3bf02" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 34397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:37:40.991829 20928 mvcc.cc:205] Tried to move back new op lower bound from 7317513260954734592 to 7317513260822007808. Current Snapshot: MvccSnapshot[applied={T|T < 7317513260954734592}]
I20260812 02:37:40.993202 20929 mvcc.cc:205] Tried to move back new op lower bound from 7317513260954734592 to 7317513260822007808. Current Snapshot: MvccSnapshot[applied={T|T < 7317513260954734592}]
I20260812 02:37:41.009900 20348 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58120:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 02:37:41.010250 20348 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58120:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 02:37:41.014266 20348 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8a7a4dc8567d46848b01831d7bd6ac4d: Sending DeleteTablet for 3 replicas of tablet 6f8cc98d5a2c49bdb9489d4fd9ca5596
I20260812 02:37:41.015568 20814 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f8cc98d5a2c49bdb9489d4fd9ca5596 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:37:41 UTC) from {username='kudu', principal='kudu/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:56240
I20260812 02:37:41.015596 20501 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f8cc98d5a2c49bdb9489d4fd9ca5596 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:37:41 UTC) from {username='kudu', principal='kudu/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:54176
I20260812 02:37:41.015789 20661 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f8cc98d5a2c49bdb9489d4fd9ca5596 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:37:41 UTC) from {username='kudu', principal='kudu/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:34540
I20260812 02:37:41.018646 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 20425
I20260812 02:37:41.038620 20941 tablet_replica.cc:330] T 6f8cc98d5a2c49bdb9489d4fd9ca5596 P 19ecf8033bb54fffa782db078c8ff845: stopping tablet replica
I20260812 02:37:41.039255 20941 raft_consensus.cc:2229] T 6f8cc98d5a2c49bdb9489d4fd9ca5596 P 19ecf8033bb54fffa782db078c8ff845 [term 1 LEADER]: Raft consensus shutting down.
I20260812 02:37:41.040133 20941 raft_consensus.cc:2258] T 6f8cc98d5a2c49bdb9489d4fd9ca5596 P 19ecf8033bb54fffa782db078c8ff845 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:37:41.040858 20939 tablet_replica.cc:330] T 6f8cc98d5a2c49bdb9489d4fd9ca5596 P 636bdd0196d7455883f3b75888d4a6ee: stopping tablet replica
I20260812 02:37:41.041091 20940 tablet_replica.cc:330] T 6f8cc98d5a2c49bdb9489d4fd9ca5596 P ba82dc6ba81f4b9d9037c2f95ed3bf02: stopping tablet replica
I20260812 02:37:41.041342 20939 raft_consensus.cc:2229] T 6f8cc98d5a2c49bdb9489d4fd9ca5596 P 636bdd0196d7455883f3b75888d4a6ee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 02:37:41.041507 20940 raft_consensus.cc:2229] T 6f8cc98d5a2c49bdb9489d4fd9ca5596 P ba82dc6ba81f4b9d9037c2f95ed3bf02 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 02:37:41.041927 20939 raft_consensus.cc:2258] T 6f8cc98d5a2c49bdb9489d4fd9ca5596 P 636bdd0196d7455883f3b75888d4a6ee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:37:41.042023 20940 raft_consensus.cc:2258] T 6f8cc98d5a2c49bdb9489d4fd9ca5596 P ba82dc6ba81f4b9d9037c2f95ed3bf02 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:37:41.044202 20940 ts_tablet_manager.cc:1905] T 6f8cc98d5a2c49bdb9489d4fd9ca5596 P ba82dc6ba81f4b9d9037c2f95ed3bf02: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:37:41.044322 20941 ts_tablet_manager.cc:1905] T 6f8cc98d5a2c49bdb9489d4fd9ca5596 P 19ecf8033bb54fffa782db078c8ff845: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 12 02:37:41 dist-test-slave-mn4r krb5kdc[20297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502258, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.132.129@KRBTEST.COM
I20260812 02:37:41.051621 20939 ts_tablet_manager.cc:1905] T 6f8cc98d5a2c49bdb9489d4fd9ca5596 P 636bdd0196d7455883f3b75888d4a6ee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:37:41.052281 20941 ts_tablet_manager.cc:1918] T 6f8cc98d5a2c49bdb9489d4fd9ca5596 P 19ecf8033bb54fffa782db078c8ff845: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 02:37:41.052563 20941 log.cc:1196] T 6f8cc98d5a2c49bdb9489d4fd9ca5596 P 19ecf8033bb54fffa782db078c8ff845: Deleting WAL directory at /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502156971023-7698-0/minicluster-data/ts-1/wal/wals/6f8cc98d5a2c49bdb9489d4fd9ca5596
I20260812 02:37:41.053313 20941 ts_tablet_manager.cc:1939] T 6f8cc98d5a2c49bdb9489d4fd9ca5596 P 19ecf8033bb54fffa782db078c8ff845: Deleting consensus metadata
I20260812 02:37:41.055589 20331 catalog_manager.cc:4841] TS 19ecf8033bb54fffa782db078c8ff845 (127.7.132.130:46065): tablet 6f8cc98d5a2c49bdb9489d4fd9ca5596 (table test-table [id=f2c4f724bf5b426e8a665e31fbcb571a]) successfully deleted
I20260812 02:37:41.057548 20940 ts_tablet_manager.cc:1918] T 6f8cc98d5a2c49bdb9489d4fd9ca5596 P ba82dc6ba81f4b9d9037c2f95ed3bf02: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 02:37:41.057828 20940 log.cc:1196] T 6f8cc98d5a2c49bdb9489d4fd9ca5596 P ba82dc6ba81f4b9d9037c2f95ed3bf02: Deleting WAL directory at /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502156971023-7698-0/minicluster-data/ts-2/wal/wals/6f8cc98d5a2c49bdb9489d4fd9ca5596
I20260812 02:37:41.057883 20907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:41.021096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54180 (local address 127.7.132.129:44803)
0812 02:37:41.021362 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:41.021372 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:41.030286 (+ 8914us) server_negotiation.cc:407] Connection header received
0812 02:37:41.030616 (+ 330us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:41.030624 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:41.030738 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:41.030831 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:37:41.033235 (+ 2404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:41.035111 (+ 1876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:41.037689 (+ 2578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:41.038664 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:41.038793 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:41.048923 (+ 10130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:37:41.048946 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:37:41.048966 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:37:41.049028 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:37:41.053177 (+ 4149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:37:41.054298 (+ 1121us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:37:41.054308 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:37:41.054312 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:37:41.054501 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:37:41.054995 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:37:41.055000 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:37:41.055004 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:37:41.055495 (+ 491us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:37:41.055820 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0812 02:37:41.057151 (+ 1331us) server_negotiation.cc:299] Negotiation successful
0812 02:37:41.057431 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 02:37:41.058523 20940 ts_tablet_manager.cc:1939] T 6f8cc98d5a2c49bdb9489d4fd9ca5596 P ba82dc6ba81f4b9d9037c2f95ed3bf02: Deleting consensus metadata
I20260812 02:37:41.058835 20571 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:37:41.060509 20333 catalog_manager.cc:4841] TS ba82dc6ba81f4b9d9037c2f95ed3bf02 (127.7.132.131:34397): tablet 6f8cc98d5a2c49bdb9489d4fd9ca5596 (table test-table [id=f2c4f724bf5b426e8a665e31fbcb571a]) successfully deleted
I20260812 02:37:41.150779 20939 ts_tablet_manager.cc:1918] T 6f8cc98d5a2c49bdb9489d4fd9ca5596 P 636bdd0196d7455883f3b75888d4a6ee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 02:37:41.151085 20939 log.cc:1196] T 6f8cc98d5a2c49bdb9489d4fd9ca5596 P 636bdd0196d7455883f3b75888d4a6ee: Deleting WAL directory at /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786502156971023-7698-0/minicluster-data/ts-0/wal/wals/6f8cc98d5a2c49bdb9489d4fd9ca5596
I20260812 02:37:41.151793 20939 ts_tablet_manager.cc:1939] T 6f8cc98d5a2c49bdb9489d4fd9ca5596 P 636bdd0196d7455883f3b75888d4a6ee: Deleting consensus metadata
I20260812 02:37:41.151885 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 20425
W20260812 02:37:41.175040 20331 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.132.129:44803 (error 108)
I20260812 02:37:41.175285 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 20584
I20260812 02:37:41.177258 20910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:41.176715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:44803 (local address 127.0.0.1:54188)
0812 02:37:41.176946 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0812 02:37:41.177074 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.129:44803: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
W20260812 02:37:41.177577 20331 catalog_manager.cc:4584] TS 636bdd0196d7455883f3b75888d4a6ee (127.7.132.129:44803): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6f8cc98d5a2c49bdb9489d4fd9ca5596: Network error: Client connection negotiation failed: client connection to 127.7.132.129:44803: connect: Connection refused (error 111)
Aug 12 02:37:41 dist-test-slave-mn4r krb5kdc[20297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502258, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.132.130@KRBTEST.COM
I20260812 02:37:41.196954 20911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:41.176264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34562 (local address 127.7.132.130:46065)
0812 02:37:41.176466 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:41.176478 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:41.176578 (+ 100us) server_negotiation.cc:407] Connection header received
0812 02:37:41.176819 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:41.176825 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:41.176913 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:41.177028 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:37:41.180160 (+ 3132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:41.181230 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:41.183389 (+ 2159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:41.183849 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:41.184054 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:41.189782 (+ 5728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:37:41.189799 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:37:41.189816 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:37:41.189862 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:37:41.193345 (+ 3483us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:37:41.194198 (+ 853us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:37:41.194211 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:37:41.194217 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:37:41.194515 (+ 298us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:37:41.195153 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:37:41.195160 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:37:41.195166 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:37:41.195639 (+ 473us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:37:41.196000 (+ 361us) server_negotiation.cc:1035] Waiting for connection context
0812 02:37:41.196288 (+ 288us) server_negotiation.cc:299] Negotiation successful
0812 02:37:41.196532 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 02:37:41.197721 20728 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:37:41.231346 20910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:41.230836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:44803 (local address 127.0.0.1:54200)
0812 02:37:41.231075 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0812 02:37:41.231185 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.129:44803: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260812 02:37:41.232506 20910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:41.232144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:44803 (local address 127.0.0.1:54212)
0812 02:37:41.232301 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0812 02:37:41.232376 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.129:44803: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260812 02:37:41.293306 20910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:41.292750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:44803 (local address 127.0.0.1:54224)
0812 02:37:41.293007 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0812 02:37:41.293117 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.129:44803: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107}
I20260812 02:37:41.294451 20910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:41.294037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:44803 (local address 127.0.0.1:54228)
0812 02:37:41.294232 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0812 02:37:41.294313 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.129:44803: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260812 02:37:41.316171 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 20584
I20260812 02:37:41.342195 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 20741
Aug 12 02:37:41 dist-test-slave-mn4r krb5kdc[20297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502258, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.132.131@KRBTEST.COM
I20260812 02:37:41.362468 20909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:41.342972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56244 (local address 127.7.132.131:34397)
0812 02:37:41.343183 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0812 02:37:41.343190 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:37:41.343284 (+ 94us) server_negotiation.cc:407] Connection header received
0812 02:37:41.343555 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:37:41.343560 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:37:41.343670 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:37:41.343786 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:37:41.345953 (+ 2167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:41.347577 (+ 1624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:41.349448 (+ 1871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:37:41.350213 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:37:41.350365 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:37:41.356452 (+ 6087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:37:41.356468 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:37:41.356485 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:37:41.356523 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:37:41.359374 (+ 2851us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:37:41.360124 (+ 750us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:37:41.360142 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:37:41.360144 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:37:41.360304 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:37:41.360716 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:37:41.360722 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:37:41.360727 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:37:41.361123 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:37:41.361453 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0812 02:37:41.361786 (+ 333us) server_negotiation.cc:299] Negotiation successful
0812 02:37:41.362050 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260812 02:37:41.363296 20887 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:37:41.375703 20910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:41.375019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:44803 (local address 127.0.0.1:54244)
0812 02:37:41.375296 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0812 02:37:41.375433 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.129:44803: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260812 02:37:41.377075 20910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:41.376535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:44803 (local address 127.0.0.1:54246)
0812 02:37:41.376801 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0812 02:37:41.376909 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.129:44803: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139}
I20260812 02:37:41.454020 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 20741
I20260812 02:37:41.477604 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 20312
I20260812 02:37:41.478899 20386 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:37:41.592720 20910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:37:41.591884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:44803 (local address 127.0.0.1:54258)
0812 02:37:41.592315 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0812 02:37:41.592473 (+ 158us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.129:44803: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":257}
I20260812 02:37:41.592875 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 20312
2026-08-12T02:37:41Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (4764 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 02:37:41 dist-test-slave-mn4r krb5kdc[20955](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 02:37:41 dist-test-slave-mn4r krb5kdc[20955](info): set up 2 sockets
Aug 12 02:37:41 dist-test-slave-mn4r krb5kdc[20955](info): commencing operation
krb5kdc: starting...
W20260812 02:37:42.828452 7698 mini_kdc.cc:121] Time spent starting KDC: real 1.183s user 0.003s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/krb5kdc/test-user.keytab.
Aug 12 02:37:43 dist-test-slave-mn4r krb5kdc[20955](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502263, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T02:37:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T02:37:43Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.7.132.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.7.132.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.7.132.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/krb5kdc/rangeradmin_127.7.132.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.7.132.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/krb5kdc/rangeradmin_127.7.132.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.7.132.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.7.132.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.7.132.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/krb5kdc/rangerlookup_127.7.132.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.7.132.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/krb5kdc/rangerlookup_127.7.132.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.7.132.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.7.132.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.7.132.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/krb5kdc/HTTP_127.7.132.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.7.132.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/krb5kdc/HTTP_127.7.132.148@KRBTEST.COM.keytab.
I20260812 02:37:43.250170 7698 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskSMpwqb/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/minicluster-data/postgres -l logfile start
2026-08-12 02:37:45.894 UTC [20988] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-12 02:37:45.894 UTC [20988] LOG: listening on IPv4 address "127.7.132.148", port 58027
2026-08-12 02:37:45.898 UTC [20988] LOG: listening on Unix socket "/tmp/.s.PGSQL.58027"
2026-08-12 02:37:45.925 UTC [20993] LOG: database system was shut down at 2026-08-12 02:37:44 UTC
2026-08-12 02:37:45.929 UTC [20988] LOG: database system is ready to accept connections
I20260812 02:37:46.999476 7698 mini_postgres.cc:96] Postgres bound to 58027
2026-08-12 02:37:47.025 UTC [21004] FATAL: database "slave" does not exist
127.7.132.148:58027 - accepting connections
I20260812 02:37:47.026434 7698 mini_ranger.cc:161] Starting Ranger...
I20260812 02:37:47.060662 7698 mini_ranger.cc:84] Created miniranger Postgres user
I20260812 02:37:47.537925 7698 mini_ranger.cc:87] Created ranger Postgres database
I20260812 02:37:47.538110 7698 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/minicluster-data/ranger-admin
2026-08-12 02:37:47,902 [I] DB FLAVOR :POSTGRES
2026-08-12 02:37:47,902 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 02:37:47,903 [I] Checking connection..
2026-08-12 02:37:47,903 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSMpwqb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskSMpwqb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.132.148:58027/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-12 02:37:48,178 [I] Checking connection passed.
2026-08-12 02:37:48,178 [I] --------- Verifying version history table ---------
2026-08-12 02:37:48,178 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSMpwqb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskSMpwqb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.132.148:58027/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 02:37:48,441 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 02:37:48,442 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSMpwqb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskSMpwqb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.132.148:58027/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 02:37:48,710 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 02:37:48,710 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-12 02:37:48,710 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSMpwqb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskSMpwqb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.132.148:58027/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskSMpwqb/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-12 02:37:48.974 UTC [21077] WARNING: there is no transaction in progress
2026-08-12 02:37:48,982 [I] create_dbversion_catalog.sql file imported successfully
2026-08-12 02:37:48,983 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSMpwqb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskSMpwqb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.132.148:58027/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 02:37:49,237 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-12 02:37:49,237 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 02:37:49,237 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSMpwqb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskSMpwqb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.132.148:58027/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-12 02:37:49,485 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSMpwqb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskSMpwqb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.132.148:58027/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-12 02:37:49,752 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSMpwqb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskSMpwqb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.132.148:58027/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-mn4r.c.gcp-upstream.internal','N') ;"
2026-08-12 02:37:49,990 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSMpwqb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskSMpwqb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.132.148:58027/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 02:37:50,259 [I] Table x_portal_user does not exist in database ranger
2026-08-12 02:37:50,259 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSMpwqb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskSMpwqb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.132.148:58027/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 02:37:50,536 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-12 02:37:50,536 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSMpwqb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskSMpwqb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.132.148:58027/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 02:37:50,776 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSMpwqb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskSMpwqb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.132.148:58027/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 02:37:51,030 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-12 02:37:51,030 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSMpwqb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskSMpwqb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.132.148:58027/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskSMpwqb/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-12 02:37:52.008 UTC [21202] WARNING: there is no transaction in progress
2026-08-12 02:37:52.026 UTC [21202] WARNING: there is no transaction in progress
2026-08-12 02:37:52.043 UTC [21202] WARNING: there is no transaction in progress
2026-08-12 02:37:52.061 UTC [21202] WARNING: there is no transaction in progress
2026-08-12 02:37:52.079 UTC [21202] WARNING: there is no transaction in progress
2026-08-12 02:37:52.098 UTC [21202] WARNING: there is no transaction in progress
2026-08-12 02:37:52.155 UTC [21202] WARNING: there is no transaction in progress
2026-08-12 02:37:52.162 UTC [21202] WARNING: there is no transaction in progress
2026-08-12 02:37:52.178 UTC [21202] WARNING: there is no transaction in progress
2026-08-12 02:37:52.188 UTC [21202] WARNING: there is no transaction in progress
2026-08-12 02:37:52.199 UTC [21202] WARNING: there is no transaction in progress
2026-08-12 02:37:52.213 UTC [21202] WARNING: there is no transaction in progress
2026-08-12 02:37:52.224 UTC [21202] WARNING: there is no transaction in progress
2026-08-12 02:37:52.231 UTC [21202] WARNING: there is no transaction in progress
2026-08-12 02:37:52.239 UTC [21202] WARNING: there is no transaction in progress
2026-08-12 02:37:52.247 UTC [21202] WARNING: there is no transaction in progress
2026-08-12 02:37:52,997 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-12 02:37:52,997 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSMpwqb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskSMpwqb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.132.148:58027/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-12 02:37:53,258 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-12 02:37:53,259 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSMpwqb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskSMpwqb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.132.148:58027/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 02:37:53,531 [I] Table x_portal_user already exists in database 'ranger'
2026-08-12 02:37:53,531 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSMpwqb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskSMpwqb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.132.148:58027/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 02:37:53,819 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-12 02:37:53,820 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSMpwqb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskSMpwqb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.132.148:58027/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 02:37:54,097 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSMpwqb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskSMpwqb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.132.148:58027/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 02:37:54,357 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSMpwqb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskSMpwqb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.132.148:58027/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-mn4r.c.gcp-upstream.internal';"
2026-08-12 02:37:54,597 [I] CORE_DB_SCHEMA import status has been updated
2026-08-12 02:37:54,597 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSMpwqb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskSMpwqb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.132.148:58027/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-12 02:37:54,867 [I] DB_PATCHES have already been applied
I20260812 02:37:54.873371 7698 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/minicluster-data/ranger-admin:/tmp/dist-test-taskSMpwqb/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskSMpwqb/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskSMpwqb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskSMpwqb/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260812 02:37:54.873467 7698 mini_ranger.cc:193] Using host: 127.7.132.148
I20260812 02:37:54.895391 7698 mini_ranger.cc:227] Ranger admin URL: http://127.7.132.148:37739
Aug 12, 2026 2:37:55 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 2:37:55 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 2:37:55 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/minicluster-data/ranger-admin/ews/webapp
Aug 12, 2026 2:37:55 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 12, 2026 2:37:55 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 12, 2026 2:37:55 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/minicluster-data/ranger-admin/ews/webapp].
Aug 12, 2026 2:37:55 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260812 02:37:56.167932 7698 mini_ranger.cc:160] Time spent starting Ranger: real 9.142s user 0.000s sys 0.064s
Aug 12, 2026 2:38:17 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 12, 2026 2:38:17 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.MetricsREST
Aug 12, 2026 2:38:17 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 12, 2026 2:38:17 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 12, 2026 2:38:17 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 12, 2026 2:38:17 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 12, 2026 2:38:17 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 12, 2026 2:38:17 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 12, 2026 2:38:17 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 12, 2026 2:38:17 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 12, 2026 2:38:17 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 12, 2026 2:38:17 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 12, 2026 2:38:17 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 12, 2026 2:38:17 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 12, 2026 2:38:17 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 12, 2026 2:38:17 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 12, 2026 2:38:17 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 12, 2026 2:38:19 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260812 02:38:19.621335 7698 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.7.132.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.7.132.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.7.132.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/krb5kdc/rangerkms_127.7.132.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.7.132.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/krb5kdc/rangerkms_127.7.132.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.7.132.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.7.132.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.7.132.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/krb5kdc/HTTP_127.7.132.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.7.132.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/krb5kdc/HTTP_127.7.132.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 12 02:38:19 dist-test-slave-mn4r krb5kdc[20955](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502299, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260812 02:38:19.856259 7698 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260812 02:38:19.891723 7698 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260812 02:38:20.919874 7698 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260812 02:38:20.920089 7698 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/minicluster-data/ranger-kms
I20260812 02:38:20.920248 7698 mini_ranger_kms.cc:227] Using postgres at 127.7.132.148:58027
2026-08-12 02:38:21,975 [I] DB FLAVOR :POSTGRES
2026-08-12 02:38:21,975 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 02:38:21,975 [I] Checking connection
2026-08-12 02:38:21,976 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSMpwqb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.132.148:58027/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-12 02:38:22,232 [I] connection success
2026-08-12 02:38:22,232 [I] --------- Verifying Ranger DB tables ---------
2026-08-12 02:38:22,232 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-12 02:38:22,233 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSMpwqb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.132.148:58027/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-12 02:38:22,511 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-12 02:38:22,512 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 02:38:22,512 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-12 02:38:22,512 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSMpwqb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.132.148:58027/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-12 02:38:22,797 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260812 02:38:23.383350 7698 mini_ranger_kms.cc:310] Created kms service
I20260812 02:38:23.589102 7698 mini_ranger_kms.cc:326] Created kudu user
I20260812 02:38:23.639401 7698 mini_ranger_kms.cc:343] Created rangerkms user
Aug 12 02:38:23 dist-test-slave-mn4r krb5kdc[20955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502299, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.7.132.148@KRBTEST.COM
I20260812 02:38:24.090420 7698 mini_ranger_kms.cc:384] Added ranger policy
I20260812 02:38:24.090790 7698 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/minicluster-data/ranger-kms:/tmp/dist-test-taskSMpwqb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskSMpwqb/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskSMpwqb/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260812 02:38:24.090855 7698 mini_ranger_kms.cc:240] Using host: 127.7.132.148
I20260812 02:38:24.117323 7698 mini_ranger_kms.cc:276] Ranger KMS PID: 21411
I20260812 02:38:24.117553 7698 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.7.132.148:40939
Aug 12, 2026 2:38:25 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 2:38:25 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 2:38:25 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 12, 2026 2:38:25 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 12, 2026 2:38:25 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/minicluster-data/ranger-kms/ews/webapp].
Aug 12, 2026 2:38:25 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 12 02:38:26 dist-test-slave-mn4r krb5kdc[20955](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786502306, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.132.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:38:26.793749 7698 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.938s user 0.170s sys 0.471s
I20260812 02:38:26.794241 7698 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260812 02:38:26.794534 7698 mini_ranger_kms.cc:401] 127.7.132.148:40939/kms/v1/keys
Aug 12 02:38:26 dist-test-slave-mn4r krb5kdc[20955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502306, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.132.148@KRBTEST.COM for HTTP/127.7.132.148@KRBTEST.COM
Aug 12 02:38:27 dist-test-slave-mn4r krb5kdc[20955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502306, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.132.148@KRBTEST.COM for HTTP/127.7.132.148@KRBTEST.COM
Aug 12 02:38:29 dist-test-slave-mn4r krb5kdc[20955](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502309, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.7.132.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
I20260812 02:38:30.047461 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:45229
--webserver_interface=127.7.132.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:36791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.132.190:45229
--ranger_config_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.132.148:40939/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:38:30.417300 21477 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:38:30.417724 21477 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:38:30.417903 21477 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:38:30.425892 21477 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:38:30.426093 21477 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:38:30.426254 21477 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:38:30.426365 21477 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:38:30.426461 21477 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:38:30.438521 21477 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:36791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/minicluster-data/master-0/wal
--ranger_kms_url=127.7.132.148:40939/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.7.132.190:45229
--ranger_config_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:45229
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.132.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:38:30.439990 21477 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:38:30.441473 21477 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:38:30.455160 21483 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:38:30.455084 21482 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:38:30.455062 21485 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:38:30.455547 21477 server_base.cc:1034] running on GCE node
I20260812 02:38:30.456648 21477 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:38:30.458325 21477 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:38:30.459579 21477 hybrid_clock.cc:648] HybridClock initialized: now 1786502310459533 us; error 41 us; skew 500 ppm
I20260812 02:38:30.459998 21477 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:38:30 dist-test-slave-mn4r krb5kdc[20955](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502310, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:38:30.463883 21477 init.cc:377] Logged in from keytab as kudu/127.7.132.190@KRBTEST.COM (short username kudu)
I20260812 02:38:30.465761 21477 webserver.cc:460] Webserver started at http://127.7.132.190:34261/ using document root <none> and password file <none>
I20260812 02:38:30.466584 21477 fs_manager.cc:359] Metadata directory not provided
I20260812 02:38:30.466740 21477 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:38:30.467092 21477 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 02:38:30 dist-test-slave-mn4r krb5kdc[20955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502310, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.190@KRBTEST.COM for HTTP/127.7.132.148@KRBTEST.COM
I20260812 02:38:30.529695 21477 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/minicluster-data/master-0/data/instance:
uuid: "166e5ea99c77472ca0c87ae3b059e9c7"
format_stamp: "Formatted at 2026-08-12 02:38:30 on dist-test-slave-mn4r"
server_key: "453b9ec30e0964bcc70f28f317666dea"
server_key_iv: "3f42ddee4dde05cf45d9c6ea10fdd45e"
server_key_version: "kuduclusterkey@0"
I20260812 02:38:30.530719 21477 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/minicluster-data/master-0/wal/instance:
uuid: "166e5ea99c77472ca0c87ae3b059e9c7"
format_stamp: "Formatted at 2026-08-12 02:38:30 on dist-test-slave-mn4r"
server_key: "453b9ec30e0964bcc70f28f317666dea"
server_key_iv: "3f42ddee4dde05cf45d9c6ea10fdd45e"
server_key_version: "kuduclusterkey@0"
I20260812 02:38:30.536819 21477 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260812 02:38:30.554994 21494 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:38:30.556690 21477 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260812 02:38:30.556895 21477 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/minicluster-data/master-0/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/minicluster-data/master-0/wal
uuid: "166e5ea99c77472ca0c87ae3b059e9c7"
format_stamp: "Formatted at 2026-08-12 02:38:30 on dist-test-slave-mn4r"
server_key: "453b9ec30e0964bcc70f28f317666dea"
server_key_iv: "3f42ddee4dde05cf45d9c6ea10fdd45e"
server_key_version: "kuduclusterkey@0"
I20260812 02:38:30.557138 21477 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:38:30.577334 21477 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:38:30.581655 21477 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:38:30.582019 21477 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:38:30.603350 21477 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.190:45229
I20260812 02:38:30.603372 21553 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.190:45229 every 8 connection(s)
I20260812 02:38:30.604918 21477 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
I20260812 02:38:30.609237 21554 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:38:30.614840 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 21477
I20260812 02:38:30.615056 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/minicluster-data/master-0/wal/instance
I20260812 02:38:30.615881 7698 external_mini_cluster.cc:1428] Setting key 6f11b4e924234e96ed2502d93d4c47c0
I20260812 02:38:30.623931 21554 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 166e5ea99c77472ca0c87ae3b059e9c7: Bootstrap starting.
I20260812 02:38:30.629855 21554 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 166e5ea99c77472ca0c87ae3b059e9c7: Neither blocks nor log segments found. Creating new log.
I20260812 02:38:30.631975 21554 log.cc:824] T 00000000000000000000000000000000 P 166e5ea99c77472ca0c87ae3b059e9c7: Log is configured to *not* fsync() on all Append() calls
I20260812 02:38:30.635768 21554 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 166e5ea99c77472ca0c87ae3b059e9c7: No bootstrap required, opened a new log
I20260812 02:38:30.645818 21554 raft_consensus.cc:348] T 00000000000000000000000000000000 P 166e5ea99c77472ca0c87ae3b059e9c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "166e5ea99c77472ca0c87ae3b059e9c7" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 45229 } }
I20260812 02:38:30.646415 21554 raft_consensus.cc:374] T 00000000000000000000000000000000 P 166e5ea99c77472ca0c87ae3b059e9c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:38:30.646553 21554 raft_consensus.cc:729] T 00000000000000000000000000000000 P 166e5ea99c77472ca0c87ae3b059e9c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 166e5ea99c77472ca0c87ae3b059e9c7, State: Initialized, Role: FOLLOWER
I20260812 02:38:30.647369 21554 consensus_queue.cc:260] T 00000000000000000000000000000000 P 166e5ea99c77472ca0c87ae3b059e9c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "166e5ea99c77472ca0c87ae3b059e9c7" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 45229 } }
I20260812 02:38:30.647655 21554 raft_consensus.cc:388] T 00000000000000000000000000000000 P 166e5ea99c77472ca0c87ae3b059e9c7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 02:38:30.647842 21554 raft_consensus.cc:482] T 00000000000000000000000000000000 P 166e5ea99c77472ca0c87ae3b059e9c7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
Aug 12 02:38:30 dist-test-slave-mn4r krb5kdc[20955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502309, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 02:38:30.648109 21554 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 166e5ea99c77472ca0c87ae3b059e9c7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:38:30.651574 21554 raft_consensus.cc:504] T 00000000000000000000000000000000 P 166e5ea99c77472ca0c87ae3b059e9c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "166e5ea99c77472ca0c87ae3b059e9c7" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 45229 } }
I20260812 02:38:30.652206 21554 leader_election.cc:302] T 00000000000000000000000000000000 P 166e5ea99c77472ca0c87ae3b059e9c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 166e5ea99c77472ca0c87ae3b059e9c7; no voters:
I20260812 02:38:30.653898 21554 leader_election.cc:288] T 00000000000000000000000000000000 P 166e5ea99c77472ca0c87ae3b059e9c7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 02:38:30.654114 21561 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 166e5ea99c77472ca0c87ae3b059e9c7 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:38:30.656307 21554 sys_catalog.cc:564] T 00000000000000000000000000000000 P 166e5ea99c77472ca0c87ae3b059e9c7 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:38:30.658576 21559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:38:30.618347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34680 (local address 127.7.132.190:45229)
0812 02:38:30.630746 (+ 12399us) server_negotiation.cc:206] Beginning negotiation
0812 02:38:30.630764 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 02:38:30.632391 (+ 1627us) server_negotiation.cc:407] Connection header received
0812 02:38:30.635190 (+ 2799us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:38:30.635225 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:38:30.635785 (+ 560us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:38:30.636309 (+ 524us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:38:30.638075 (+ 1766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:38:30.640306 (+ 2231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:38:30.642420 (+ 2114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:38:30.643623 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:38:30.643848 (+ 225us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:38:30.649344 (+ 5496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:38:30.649354 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:38:30.649371 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:38:30.649427 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:38:30.652771 (+ 3344us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:38:30.653892 (+ 1121us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:38:30.653901 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:38:30.653910 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:38:30.654170 (+ 260us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:38:30.654727 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:38:30.654732 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:38:30.654735 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:38:30.655358 (+ 623us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:38:30.655697 (+ 339us) server_negotiation.cc:1035] Waiting for connection context
0812 02:38:30.656188 (+ 491us) server_negotiation.cc:299] Negotiation successful
0812 02:38:30.656673 (+ 485us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12121,"thread_start_us":11849,"threads_started":1}
I20260812 02:38:30.663602 21561 raft_consensus.cc:686] T 00000000000000000000000000000000 P 166e5ea99c77472ca0c87ae3b059e9c7 [term 1 LEADER]: Becoming Leader. State: Replica: 166e5ea99c77472ca0c87ae3b059e9c7, State: Running, Role: LEADER
I20260812 02:38:30.664492 21561 consensus_queue.cc:237] T 00000000000000000000000000000000 P 166e5ea99c77472ca0c87ae3b059e9c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "166e5ea99c77472ca0c87ae3b059e9c7" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 45229 } }
I20260812 02:38:30.668195 21566 sys_catalog.cc:455] T 00000000000000000000000000000000 P 166e5ea99c77472ca0c87ae3b059e9c7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 166e5ea99c77472ca0c87ae3b059e9c7. Latest consensus state: current_term: 1 leader_uuid: "166e5ea99c77472ca0c87ae3b059e9c7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "166e5ea99c77472ca0c87ae3b059e9c7" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 45229 } } }
I20260812 02:38:30.668221 21562 sys_catalog.cc:455] T 00000000000000000000000000000000 P 166e5ea99c77472ca0c87ae3b059e9c7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "166e5ea99c77472ca0c87ae3b059e9c7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "166e5ea99c77472ca0c87ae3b059e9c7" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 45229 } } }
I20260812 02:38:30.668691 21566 sys_catalog.cc:458] T 00000000000000000000000000000000 P 166e5ea99c77472ca0c87ae3b059e9c7 [sys.catalog]: This master's current role is: LEADER
I20260812 02:38:30.668691 21562 sys_catalog.cc:458] T 00000000000000000000000000000000 P 166e5ea99c77472ca0c87ae3b059e9c7 [sys.catalog]: This master's current role is: LEADER
I20260812 02:38:30.669615 21571 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:38:30.682324 21571 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 02:38:30.714864 21571 catalog_manager.cc:1347] Generated new cluster ID: 4bcf8ee7ae6148648d433c350e389b2b
I20260812 02:38:30.715197 21571 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 02:38:30.721028 21554 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260812 02:38:30.772341 21571 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 02:38:30.782068 21571 catalog_manager.cc:1504] Loading token signing keys...
I20260812 02:38:30.803177 21571 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 166e5ea99c77472ca0c87ae3b059e9c7: Generated new TSK 0
I20260812 02:38:30.805089 21571 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.132.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.129.keytab.
Entry for principal kudu/127.7.132.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.129.keytab.
I20260812 02:38:31.843238 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.132.148:40939/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.129:0
--local_ip_for_outbound_sockets=127.7.132.129
--webserver_interface=127.7.132.129
--webserver_port=0
--tserver_master_addrs=127.7.132.190:45229
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.129.keytab
--principal=kudu/127.7.132.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:36791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 02:38:32 dist-test-slave-mn4r krb5kdc[20955](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786502312, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260812 02:38:32.298653 21609 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:38:32.299024 21609 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:38:32.299201 21609 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:38:32.307216 21609 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:38:32.307459 21609 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:38:32.307647 21609 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.129
I20260812 02:38:32.318219 21609 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:36791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.7.132.148:40939/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.129.keytab
--principal=kudu/127.7.132.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.132.129
--webserver_port=0
--tserver_master_addrs=127.7.132.190:45229
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.129
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:38:32.319713 21609 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:38:32.321404 21609 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:38:32.341109 21622 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:38:32.341990 21621 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:38:32.347286 21624 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:38:32.347642 21609 server_base.cc:1034] running on GCE node
I20260812 02:38:32.348572 21609 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:38:32.354168 21609 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:38:32.355530 21609 hybrid_clock.cc:648] HybridClock initialized: now 1786502312355434 us; error 86 us; skew 500 ppm
I20260812 02:38:32.356058 21609 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:38:32 dist-test-slave-mn4r krb5kdc[20955](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502312, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:38:32.360935 21609 init.cc:377] Logged in from keytab as kudu/127.7.132.129@KRBTEST.COM (short username kudu)
I20260812 02:38:32.363950 21609 webserver.cc:460] Webserver started at http://127.7.132.129:43007/ using document root <none> and password file <none>
I20260812 02:38:32.365060 21609 fs_manager.cc:359] Metadata directory not provided
I20260812 02:38:32.365293 21609 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:38:32.365795 21609 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 02:38:32 dist-test-slave-mn4r krb5kdc[20955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502312, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.129@KRBTEST.COM for HTTP/127.7.132.148@KRBTEST.COM
I20260812 02:38:32.386170 21609 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/minicluster-data/ts-0/data/instance:
uuid: "9eaa55ad48b3402480437fe9868a38cd"
format_stamp: "Formatted at 2026-08-12 02:38:32 on dist-test-slave-mn4r"
server_key: "56f5407f999d0d481e659f50219306b9"
server_key_iv: "2f218a98ecd5a81d69ce4f4e18da12d2"
server_key_version: "kuduclusterkey@0"
I20260812 02:38:32.387531 21609 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/minicluster-data/ts-0/wal/instance:
uuid: "9eaa55ad48b3402480437fe9868a38cd"
format_stamp: "Formatted at 2026-08-12 02:38:32 on dist-test-slave-mn4r"
server_key: "56f5407f999d0d481e659f50219306b9"
server_key_iv: "2f218a98ecd5a81d69ce4f4e18da12d2"
server_key_version: "kuduclusterkey@0"
I20260812 02:38:32.395777 21609 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.006s sys 0.000s
I20260812 02:38:32.416234 21631 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:38:32.418483 21609 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.005s sys 0.000s
I20260812 02:38:32.418699 21609 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/minicluster-data/ts-0/wal
uuid: "9eaa55ad48b3402480437fe9868a38cd"
format_stamp: "Formatted at 2026-08-12 02:38:32 on dist-test-slave-mn4r"
server_key: "56f5407f999d0d481e659f50219306b9"
server_key_iv: "2f218a98ecd5a81d69ce4f4e18da12d2"
server_key_version: "kuduclusterkey@0"
I20260812 02:38:32.419013 21609 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:38:32.470014 21609 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:38:32.475201 21609 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:38:32.475781 21609 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:38:32.478118 21609 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:38:32.480490 21609 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:38:32.480607 21609 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.001s user 0.000s sys 0.001s
I20260812 02:38:32.480749 21609 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:38:32.480827 21609 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:38:32.537014 21609 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.129:43585
I20260812 02:38:32.537061 21766 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.129:43585 every 8 connection(s)
I20260812 02:38:32.539013 21609 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/minicluster-data/ts-0/data/info.pb
I20260812 02:38:32.547199 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 21609
I20260812 02:38:32.547437 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/minicluster-data/ts-0/wal/instance
I20260812 02:38:32.548167 7698 external_mini_cluster.cc:1428] Setting key 7cdf6a55b3b72762344fb57a0bb92c93
Aug 12 02:38:32 dist-test-slave-mn4r krb5kdc[20955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502312, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.129@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.132.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.130@KRBTEST.COM" created.
I20260812 02:38:32.596218 21770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:38:32.544914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:52353 (local address 127.7.132.190:45229)
0812 02:38:32.566511 (+ 21597us) server_negotiation.cc:206] Beginning negotiation
0812 02:38:32.566519 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:38:32.567877 (+ 1358us) server_negotiation.cc:407] Connection header received
0812 02:38:32.570201 (+ 2324us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:38:32.570207 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:38:32.570318 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:38:32.570403 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:38:32.572821 (+ 2418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:38:32.574020 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:38:32.576622 (+ 2602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:38:32.577295 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:38:32.577392 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:38:32.587845 (+ 10453us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:38:32.587853 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:38:32.587858 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:38:32.587928 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:38:32.591634 (+ 3706us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:38:32.592954 (+ 1320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:38:32.592963 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:38:32.592969 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:38:32.593160 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:38:32.593805 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:38:32.593813 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:38:32.593819 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:38:32.594262 (+ 443us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:38:32.594389 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0812 02:38:32.595549 (+ 1160us) server_negotiation.cc:299] Negotiation successful
0812 02:38:32.595806 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21448,"thread_start_us":21321,"threads_started":1}
I20260812 02:38:32.598683 21769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:38:32.542906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:45229 (local address 127.7.132.129:52353)
0812 02:38:32.567201 (+ 24295us) negotiation.cc:107] Waiting for socket to connect
0812 02:38:32.567265 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0812 02:38:32.569789 (+ 2524us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:38:32.571051 (+ 1262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:38:32.571064 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:38:32.571725 (+ 661us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:38:32.572596 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:38:32.572622 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:38:32.574297 (+ 1675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:38:32.574303 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:38:32.576426 (+ 2123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:38:32.576442 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:38:32.581260 (+ 4818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:38:32.581269 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:38:32.581603 (+ 334us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:38:32.583793 (+ 2190us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:38:32.583827 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:38:32.587518 (+ 3691us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:38:32.591911 (+ 4393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:38:32.591923 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:38:32.591944 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:38:32.592626 (+ 682us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:38:32.593335 (+ 709us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:38:32.593339 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:38:32.593342 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:38:32.593643 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:38:32.594510 (+ 867us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:38:32.594518 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:38:32.595036 (+ 518us) client_negotiation.cc:769] Sending connection context
0812 02:38:32.596470 (+ 1434us) client_negotiation.cc:240] Negotiation successful
0812 02:38:32.597113 (+ 643us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23914,"thread_start_us":23669,"threads_started":1}
I20260812 02:38:32.602304 21767 heartbeater.cc:347] Connected to a master server at 127.7.132.190:45229
I20260812 02:38:32.602669 21767 heartbeater.cc:464] Registering TS with master...
I20260812 02:38:32.603623 21767 heartbeater.cc:511] Master 127.7.132.190:45229 requested a full tablet report, sending...
I20260812 02:38:32.606518 21514 ts_manager.cc:194] Registered new tserver with Master: 9eaa55ad48b3402480437fe9868a38cd (127.7.132.129:43585)
I20260812 02:38:32.608935 21514 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.129@KRBTEST.COM'} at 127.7.132.129:52353
Aug 12 02:38:32 dist-test-slave-mn4r krb5kdc[20955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502312, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.190@KRBTEST.COM for HTTP/127.7.132.148@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.130.keytab.
Entry for principal kudu/127.7.132.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.130.keytab.
I20260812 02:38:32.648572 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.132.148:40939/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.130:0
--local_ip_for_outbound_sockets=127.7.132.130
--webserver_interface=127.7.132.130
--webserver_port=0
--tserver_master_addrs=127.7.132.190:45229
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.130.keytab
--principal=kudu/127.7.132.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:36791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 02:38:32 dist-test-slave-mn4r krb5kdc[20955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502312, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.190@KRBTEST.COM for HTTP/127.7.132.148@KRBTEST.COM
W20260812 02:38:33.135604 21774 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:38:33.136152 21774 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:38:33.136385 21774 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:38:33.148496 21774 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:38:33.148660 21774 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:38:33.148909 21774 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.130
I20260812 02:38:33.165186 21774 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:36791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.7.132.148:40939/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.130.keytab
--principal=kudu/127.7.132.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.132.130
--webserver_port=0
--tserver_master_addrs=127.7.132.190:45229
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.130
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:38:33.167212 21774 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:38:33.169571 21774 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:38:33.195923 21782 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:38:33.195950 21783 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:38:33.203728 21786 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:38:33.204314 21774 server_base.cc:1034] running on GCE node
I20260812 02:38:33.205216 21774 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:38:33.208734 21774 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:38:33.210232 21774 hybrid_clock.cc:648] HybridClock initialized: now 1786502313210122 us; error 109 us; skew 500 ppm
I20260812 02:38:33.210662 21774 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:38:33 dist-test-slave-mn4r krb5kdc[20955](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502313, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:38:33.215308 21774 init.cc:377] Logged in from keytab as kudu/127.7.132.130@KRBTEST.COM (short username kudu)
I20260812 02:38:33.230041 21774 webserver.cc:460] Webserver started at http://127.7.132.130:37083/ using document root <none> and password file <none>
I20260812 02:38:33.231359 21774 fs_manager.cc:359] Metadata directory not provided
I20260812 02:38:33.231524 21774 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:38:33.231935 21774 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 02:38:33 dist-test-slave-mn4r krb5kdc[20955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502313, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.130@KRBTEST.COM for HTTP/127.7.132.148@KRBTEST.COM
I20260812 02:38:33.252592 21774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/minicluster-data/ts-1/data/instance:
uuid: "2de9075232854ebca6189aa90d928a11"
format_stamp: "Formatted at 2026-08-12 02:38:33 on dist-test-slave-mn4r"
server_key: "43f75cc6c9c5fe6db10b5f147d3ee685"
server_key_iv: "75ad16bc7c22cf51d9c60db3b039239d"
server_key_version: "kuduclusterkey@0"
I20260812 02:38:33.254143 21774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/minicluster-data/ts-1/wal/instance:
uuid: "2de9075232854ebca6189aa90d928a11"
format_stamp: "Formatted at 2026-08-12 02:38:33 on dist-test-slave-mn4r"
server_key: "43f75cc6c9c5fe6db10b5f147d3ee685"
server_key_iv: "75ad16bc7c22cf51d9c60db3b039239d"
server_key_version: "kuduclusterkey@0"
I20260812 02:38:33.275279 21774 fs_manager.cc:691] Time spent creating directory manager: real 0.021s user 0.006s sys 0.011s
I20260812 02:38:33.297916 21804 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:38:33.299988 21774 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260812 02:38:33.300204 21774 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/minicluster-data/ts-1/wal
uuid: "2de9075232854ebca6189aa90d928a11"
format_stamp: "Formatted at 2026-08-12 02:38:33 on dist-test-slave-mn4r"
server_key: "43f75cc6c9c5fe6db10b5f147d3ee685"
server_key_iv: "75ad16bc7c22cf51d9c60db3b039239d"
server_key_version: "kuduclusterkey@0"
I20260812 02:38:33.300513 21774 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:38:33.321098 21774 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:38:33.326540 21774 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:38:33.326917 21774 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:38:33.328248 21774 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:38:33.330287 21774 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:38:33.330399 21774 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:38:33.330529 21774 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:38:33.330595 21774 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:38:33.392748 21774 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.130:33885
I20260812 02:38:33.392783 21933 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.130:33885 every 8 connection(s)
I20260812 02:38:33.394212 21774 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/minicluster-data/ts-1/data/info.pb
I20260812 02:38:33.396338 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 21774
I20260812 02:38:33.396528 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/minicluster-data/ts-1/wal/instance
I20260812 02:38:33.397109 7698 external_mini_cluster.cc:1428] Setting key 69dd76ece3efd4479b21753e5714ccaf
Aug 12 02:38:33 dist-test-slave-mn4r krb5kdc[20955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502313, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.130@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.132.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.131@KRBTEST.COM" created.
I20260812 02:38:33.433033 21937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:38:33.397118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:52659 (local address 127.7.132.190:45229)
0812 02:38:33.398419 (+ 1301us) server_negotiation.cc:206] Beginning negotiation
0812 02:38:33.398426 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:38:33.399263 (+ 837us) server_negotiation.cc:407] Connection header received
0812 02:38:33.402419 (+ 3156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:38:33.402427 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:38:33.402521 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:38:33.402645 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:38:33.404648 (+ 2003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:38:33.405947 (+ 1299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:38:33.408483 (+ 2536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:38:33.409211 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:38:33.409360 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:38:33.421997 (+ 12637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:38:33.422005 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:38:33.422008 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:38:33.422055 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:38:33.428562 (+ 6507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:38:33.429599 (+ 1037us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:38:33.429609 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:38:33.429612 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:38:33.429801 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:38:33.430567 (+ 766us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:38:33.430572 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:38:33.430576 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:38:33.431036 (+ 460us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:38:33.431231 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0812 02:38:33.432536 (+ 1305us) server_negotiation.cc:299] Negotiation successful
0812 02:38:33.432747 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1166,"thread_start_us":1307,"threads_started":1}
I20260812 02:38:33.434862 21938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:38:33.397974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:45229 (local address 127.7.132.130:52659)
0812 02:38:33.399092 (+ 1118us) negotiation.cc:107] Waiting for socket to connect
0812 02:38:33.399154 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0812 02:38:33.402025 (+ 2871us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:38:33.402934 (+ 909us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:38:33.402948 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:38:33.403591 (+ 643us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:38:33.404406 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:38:33.404430 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:38:33.406144 (+ 1714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:38:33.406150 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:38:33.408324 (+ 2174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:38:33.408340 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:38:33.409472 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:38:33.409480 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:38:33.409812 (+ 332us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:38:33.419040 (+ 9228us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:38:33.419071 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:38:33.421662 (+ 2591us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:38:33.428817 (+ 7155us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:38:33.428828 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:38:33.428844 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:38:33.429432 (+ 588us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:38:33.429961 (+ 529us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:38:33.429964 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:38:33.429967 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:38:33.430397 (+ 430us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:38:33.431330 (+ 933us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:38:33.431343 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:38:33.431905 (+ 562us) client_negotiation.cc:769] Sending connection context
0812 02:38:33.432336 (+ 431us) client_negotiation.cc:240] Negotiation successful
0812 02:38:33.433036 (+ 700us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":810,"thread_start_us":545,"threads_started":1}
I20260812 02:38:33.436733 21934 heartbeater.cc:347] Connected to a master server at 127.7.132.190:45229
I20260812 02:38:33.437089 21934 heartbeater.cc:464] Registering TS with master...
I20260812 02:38:33.437872 21934 heartbeater.cc:511] Master 127.7.132.190:45229 requested a full tablet report, sending...
I20260812 02:38:33.439802 21514 ts_manager.cc:194] Registered new tserver with Master: 2de9075232854ebca6189aa90d928a11 (127.7.132.130:33885)
I20260812 02:38:33.441037 21514 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.130@KRBTEST.COM'} at 127.7.132.130:52659
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.131.keytab.
Entry for principal kudu/127.7.132.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.131.keytab.
I20260812 02:38:33.470057 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.132.148:40939/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.131:0
--local_ip_for_outbound_sockets=127.7.132.131
--webserver_interface=127.7.132.131
--webserver_port=0
--tserver_master_addrs=127.7.132.190:45229
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.131.keytab
--principal=kudu/127.7.132.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:36791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 02:38:33.613446 21767 heartbeater.cc:503] Master 127.7.132.190:45229 was elected leader, sending a full tablet report...
W20260812 02:38:33.828413 21943 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:38:33.828754 21943 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:38:33.828917 21943 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:38:33.836545 21943 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:38:33.836706 21943 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:38:33.836884 21943 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.131
I20260812 02:38:33.847390 21943 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:36791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.7.132.148:40939/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.131.keytab
--principal=kudu/127.7.132.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.132.131
--webserver_port=0
--tserver_master_addrs=127.7.132.190:45229
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.131
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:38:33.848773 21943 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:38:33.850286 21943 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:38:33.864517 21949 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:38:33.864609 21951 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:38:33.864607 21948 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:38:33.867532 21943 server_base.cc:1034] running on GCE node
I20260812 02:38:33.868217 21943 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:38:33.869251 21943 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:38:33.870479 21943 hybrid_clock.cc:648] HybridClock initialized: now 1786502313870441 us; error 53 us; skew 500 ppm
I20260812 02:38:33.870808 21943 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:38:33 dist-test-slave-mn4r krb5kdc[20955](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502313, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:38:33.874013 21943 init.cc:377] Logged in from keytab as kudu/127.7.132.131@KRBTEST.COM (short username kudu)
I20260812 02:38:33.875592 21943 webserver.cc:460] Webserver started at http://127.7.132.131:43511/ using document root <none> and password file <none>
I20260812 02:38:33.876322 21943 fs_manager.cc:359] Metadata directory not provided
I20260812 02:38:33.876468 21943 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:38:33.876793 21943 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 02:38:33 dist-test-slave-mn4r krb5kdc[20955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502313, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.131@KRBTEST.COM for HTTP/127.7.132.148@KRBTEST.COM
I20260812 02:38:33.890919 21943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/minicluster-data/ts-2/data/instance:
uuid: "414e120080594464900790a1e1c6b621"
format_stamp: "Formatted at 2026-08-12 02:38:33 on dist-test-slave-mn4r"
server_key: "b87807811067fa96260e1f4590df8f89"
server_key_iv: "80fc271d4e6008bd56bf5d5bcf73d844"
server_key_version: "kuduclusterkey@0"
I20260812 02:38:33.891832 21943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/minicluster-data/ts-2/wal/instance:
uuid: "414e120080594464900790a1e1c6b621"
format_stamp: "Formatted at 2026-08-12 02:38:33 on dist-test-slave-mn4r"
server_key: "b87807811067fa96260e1f4590df8f89"
server_key_iv: "80fc271d4e6008bd56bf5d5bcf73d844"
server_key_version: "kuduclusterkey@0"
I20260812 02:38:33.897531 21943 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260812 02:38:33.915668 21958 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:38:33.917714 21943 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260812 02:38:33.917938 21943 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/minicluster-data/ts-2/wal
uuid: "414e120080594464900790a1e1c6b621"
format_stamp: "Formatted at 2026-08-12 02:38:33 on dist-test-slave-mn4r"
server_key: "b87807811067fa96260e1f4590df8f89"
server_key_iv: "80fc271d4e6008bd56bf5d5bcf73d844"
server_key_version: "kuduclusterkey@0"
I20260812 02:38:33.918254 21943 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:38:33.935705 21943 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:38:33.939654 21943 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:38:33.939936 21943 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:38:33.940877 21943 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:38:33.942360 21943 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:38:33.942436 21943 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:38:33.942538 21943 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:38:33.942586 21943 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:38:33.976931 21943 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.131:40599
I20260812 02:38:33.977026 22099 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.131:40599 every 8 connection(s)
I20260812 02:38:33.978426 21943 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/minicluster-data/ts-2/data/info.pb
I20260812 02:38:33.979142 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 21943
I20260812 02:38:33.979357 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/minicluster-data/ts-2/wal/instance
I20260812 02:38:33.979928 7698 external_mini_cluster.cc:1428] Setting key 92522dab3a4dd0bc0c24356fbaf5a5a3
Aug 12 02:38:33 dist-test-slave-mn4r krb5kdc[20955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502313, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.131@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 02:38:34.000527 22102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:38:33.981404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:37587 (local address 127.7.132.190:45229)
0812 02:38:33.981959 (+ 555us) server_negotiation.cc:206] Beginning negotiation
0812 02:38:33.981966 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:38:33.983369 (+ 1403us) server_negotiation.cc:407] Connection header received
0812 02:38:33.985437 (+ 2068us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:38:33.985442 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:38:33.985549 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:38:33.985681 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:38:33.987301 (+ 1620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:38:33.988052 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:38:33.989726 (+ 1674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:38:33.990292 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:38:33.990423 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:38:33.993960 (+ 3537us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:38:33.993965 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:38:33.993968 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:38:33.994019 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:38:33.997232 (+ 3213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:38:33.998026 (+ 794us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:38:33.998032 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:38:33.998036 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:38:33.998211 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:38:33.998681 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:38:33.998688 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:38:33.998691 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:38:33.998981 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:38:33.999131 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0812 02:38:33.999908 (+ 777us) server_negotiation.cc:299] Negotiation successful
0812 02:38:34.000145 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":405,"thread_start_us":245,"threads_started":1}
I20260812 02:38:34.001621 22104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:38:33.982204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:45229 (local address 127.7.132.131:37587)
0812 02:38:33.983188 (+ 984us) negotiation.cc:107] Waiting for socket to connect
0812 02:38:33.983264 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0812 02:38:33.985160 (+ 1896us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:38:33.985944 (+ 784us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:38:33.985954 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:38:33.986500 (+ 546us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:38:33.987128 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:38:33.987148 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:38:33.988238 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:38:33.988242 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:38:33.989537 (+ 1295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:38:33.989551 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:38:33.990540 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:38:33.990548 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:38:33.990809 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:38:33.991549 (+ 740us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:38:33.991579 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:38:33.993697 (+ 2118us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:38:33.997379 (+ 3682us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:38:33.997389 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:38:33.997407 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:38:33.997902 (+ 495us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:38:33.998328 (+ 426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:38:33.998334 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:38:33.998338 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:38:33.998565 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:38:33.999173 (+ 608us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:38:33.999181 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:38:33.999533 (+ 352us) client_negotiation.cc:769] Sending connection context
0812 02:38:33.999842 (+ 309us) client_negotiation.cc:240] Negotiation successful
0812 02:38:34.000415 (+ 573us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":609,"thread_start_us":366,"threads_started":1}
I20260812 02:38:34.003072 22100 heartbeater.cc:347] Connected to a master server at 127.7.132.190:45229
I20260812 02:38:34.003388 22100 heartbeater.cc:464] Registering TS with master...
I20260812 02:38:34.004168 22100 heartbeater.cc:511] Master 127.7.132.190:45229 requested a full tablet report, sending...
I20260812 02:38:34.005617 21514 ts_manager.cc:194] Registered new tserver with Master: 414e120080594464900790a1e1c6b621 (127.7.132.131:40599)
I20260812 02:38:34.006633 21514 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.131@KRBTEST.COM'} at 127.7.132.131:37587
I20260812 02:38:34.008819 7698 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 02:38:34.026662 22102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:38:34.013389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34682 (local address 127.7.132.190:45229)
0812 02:38:34.013607 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0812 02:38:34.013612 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:38:34.014140 (+ 528us) server_negotiation.cc:407] Connection header received
0812 02:38:34.014327 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:38:34.014331 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:38:34.014407 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:38:34.014494 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:38:34.015943 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:38:34.016644 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:38:34.018027 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:38:34.018569 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:38:34.018677 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:38:34.021064 (+ 2387us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:38:34.021070 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:38:34.021074 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:38:34.021109 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:38:34.024006 (+ 2897us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:38:34.024730 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:38:34.024734 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:38:34.024736 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:38:34.024895 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:38:34.025312 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:38:34.025316 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:38:34.025318 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:38:34.025659 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:38:34.025854 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0812 02:38:34.026230 (+ 376us) server_negotiation.cc:299] Negotiation successful
0812 02:38:34.026434 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 02:38:34.030813 21514 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34682:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 02:38:34.038269 21514 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 02:38:34.074862 22120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:38:34.052123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54216 (local address 127.7.132.129:43585)
0812 02:38:34.064683 (+ 12560us) server_negotiation.cc:206] Beginning negotiation
0812 02:38:34.064698 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 02:38:34.066485 (+ 1787us) server_negotiation.cc:407] Connection header received
0812 02:38:34.066585 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:38:34.066591 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:38:34.066832 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:38:34.067037 (+ 205us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:38:34.068689 (+ 1652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:38:34.070184 (+ 1495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:38:34.072521 (+ 2337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:38:34.073534 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:38:34.073700 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:38:34.073965 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0812 02:38:34.074326 (+ 361us) server_negotiation.cc:299] Negotiation successful
0812 02:38:34.074501 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12289,"thread_start_us":13051,"threads_started":1}
I20260812 02:38:34.074980 22117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:38:34.051421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:43585 (local address 127.0.0.1:54216)
0812 02:38:34.065960 (+ 14539us) negotiation.cc:107] Waiting for socket to connect
0812 02:38:34.066038 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0812 02:38:34.066262 (+ 224us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:38:34.067352 (+ 1090us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:38:34.067360 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:38:34.067404 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:38:34.068502 (+ 1098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:38:34.068517 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:38:34.070363 (+ 1846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:38:34.070367 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:38:34.072332 (+ 1965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:38:34.072347 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:38:34.073784 (+ 1437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:38:34.073791 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:38:34.074027 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:38:34.074070 (+ 43us) client_negotiation.cc:769] Sending connection context
0812 02:38:34.074265 (+ 195us) client_negotiation.cc:240] Negotiation successful
0812 02:38:34.074491 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14212,"mutex_wait_us":601,"spinlock_wait_cycles":3328,"thread_start_us":12189,"threads_started":1}
I20260812 02:38:34.075966 22119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:38:34.052038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40112 (local address 127.7.132.131:40599)
0812 02:38:34.065244 (+ 13206us) server_negotiation.cc:206] Beginning negotiation
0812 02:38:34.065253 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 02:38:34.066181 (+ 928us) server_negotiation.cc:407] Connection header received
0812 02:38:34.066516 (+ 335us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:38:34.066526 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:38:34.066845 (+ 319us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:38:34.067036 (+ 191us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:38:34.068331 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:38:34.070009 (+ 1678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:38:34.072617 (+ 2608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:38:34.074161 (+ 1544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:38:34.074307 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:38:34.074565 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0812 02:38:34.075378 (+ 813us) server_negotiation.cc:299] Negotiation successful
0812 02:38:34.075565 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13027,"thread_start_us":12848,"threads_started":1}
I20260812 02:38:34.076942 22118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:38:34.051651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.131:40599 (local address 127.0.0.1:40112)
0812 02:38:34.065960 (+ 14309us) negotiation.cc:107] Waiting for socket to connect
0812 02:38:34.066038 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0812 02:38:34.066271 (+ 233us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:38:34.067352 (+ 1081us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:38:34.067360 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:38:34.067404 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:38:34.068023 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:38:34.068042 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:38:34.070205 (+ 2163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:38:34.070210 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:38:34.072408 (+ 2198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:38:34.072453 (+ 45us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:38:34.074739 (+ 2286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:38:34.074745 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:38:34.074925 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:38:34.074959 (+ 34us) client_negotiation.cc:769] Sending connection context
0812 02:38:34.075053 (+ 94us) client_negotiation.cc:240] Negotiation successful
0812 02:38:34.076512 (+ 1459us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14065,"mutex_wait_us":73,"spinlock_wait_cycles":16000,"thread_start_us":13854,"threads_started":1}
I20260812 02:38:34.077193 22122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:38:34.066058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.130:33885 (local address 127.0.0.1:48818)
0812 02:38:34.068385 (+ 2327us) negotiation.cc:107] Waiting for socket to connect
0812 02:38:34.068402 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 02:38:34.068581 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:38:34.069492 (+ 911us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:38:34.069498 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:38:34.069523 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:38:34.069925 (+ 402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:38:34.069940 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:38:34.071938 (+ 1998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:38:34.071944 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:38:34.074174 (+ 2230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:38:34.074188 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:38:34.076325 (+ 2137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:38:34.076334 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:38:34.076510 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:38:34.076541 (+ 31us) client_negotiation.cc:769] Sending connection context
0812 02:38:34.076647 (+ 106us) client_negotiation.cc:240] Negotiation successful
0812 02:38:34.076788 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2202,"thread_start_us":2013,"threads_started":1}
I20260812 02:38:34.077196 22121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:38:34.066063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48818 (local address 127.7.132.130:33885)
0812 02:38:34.067909 (+ 1846us) server_negotiation.cc:206] Beginning negotiation
0812 02:38:34.067918 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 02:38:34.068526 (+ 608us) server_negotiation.cc:407] Connection header received
0812 02:38:34.068787 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:38:34.068795 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:38:34.069049 (+ 254us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:38:34.069268 (+ 219us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:38:34.070205 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:38:34.071781 (+ 1576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:38:34.074360 (+ 2579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:38:34.075817 (+ 1457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:38:34.075955 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:38:34.076166 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0812 02:38:34.076721 (+ 555us) server_negotiation.cc:299] Negotiation successful
0812 02:38:34.076902 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1645,"thread_start_us":1522,"threads_started":1}
I20260812 02:38:34.080040 21690 tablet_service.cc:1467] Processing CreateTablet for tablet 484ed45ef3674c4ca25cf7c625d69e1c (DEFAULT_TABLE table=test-table [id=4dede43767a548f8a0ad766947a5f587]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:38:34.080382 22020 tablet_service.cc:1467] Processing CreateTablet for tablet 484ed45ef3674c4ca25cf7c625d69e1c (DEFAULT_TABLE table=test-table [id=4dede43767a548f8a0ad766947a5f587]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:38:34.080951 21859 tablet_service.cc:1467] Processing CreateTablet for tablet 484ed45ef3674c4ca25cf7c625d69e1c (DEFAULT_TABLE table=test-table [id=4dede43767a548f8a0ad766947a5f587]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:38:34.081770 22020 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 484ed45ef3674c4ca25cf7c625d69e1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:38:34.081779 21690 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 484ed45ef3674c4ca25cf7c625d69e1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:38:34.082347 21859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 484ed45ef3674c4ca25cf7c625d69e1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:38:34.094185 22124 tablet_bootstrap.cc:492] T 484ed45ef3674c4ca25cf7c625d69e1c P 414e120080594464900790a1e1c6b621: Bootstrap starting.
I20260812 02:38:34.097206 22125 tablet_bootstrap.cc:492] T 484ed45ef3674c4ca25cf7c625d69e1c P 9eaa55ad48b3402480437fe9868a38cd: Bootstrap starting.
I20260812 02:38:34.098102 22124 tablet_bootstrap.cc:654] T 484ed45ef3674c4ca25cf7c625d69e1c P 414e120080594464900790a1e1c6b621: Neither blocks nor log segments found. Creating new log.
I20260812 02:38:34.098109 22126 tablet_bootstrap.cc:492] T 484ed45ef3674c4ca25cf7c625d69e1c P 2de9075232854ebca6189aa90d928a11: Bootstrap starting.
I20260812 02:38:34.099956 22124 log.cc:824] T 484ed45ef3674c4ca25cf7c625d69e1c P 414e120080594464900790a1e1c6b621: Log is configured to *not* fsync() on all Append() calls
I20260812 02:38:34.101854 22126 tablet_bootstrap.cc:654] T 484ed45ef3674c4ca25cf7c625d69e1c P 2de9075232854ebca6189aa90d928a11: Neither blocks nor log segments found. Creating new log.
I20260812 02:38:34.103299 22126 log.cc:824] T 484ed45ef3674c4ca25cf7c625d69e1c P 2de9075232854ebca6189aa90d928a11: Log is configured to *not* fsync() on all Append() calls
I20260812 02:38:34.103675 22125 tablet_bootstrap.cc:654] T 484ed45ef3674c4ca25cf7c625d69e1c P 9eaa55ad48b3402480437fe9868a38cd: Neither blocks nor log segments found. Creating new log.
I20260812 02:38:34.104712 22124 tablet_bootstrap.cc:492] T 484ed45ef3674c4ca25cf7c625d69e1c P 414e120080594464900790a1e1c6b621: No bootstrap required, opened a new log
I20260812 02:38:34.105041 22125 log.cc:824] T 484ed45ef3674c4ca25cf7c625d69e1c P 9eaa55ad48b3402480437fe9868a38cd: Log is configured to *not* fsync() on all Append() calls
I20260812 02:38:34.105086 22124 ts_tablet_manager.cc:1397] T 484ed45ef3674c4ca25cf7c625d69e1c P 414e120080594464900790a1e1c6b621: Time spent bootstrapping tablet: real 0.011s user 0.009s sys 0.000s
I20260812 02:38:34.106870 22126 tablet_bootstrap.cc:492] T 484ed45ef3674c4ca25cf7c625d69e1c P 2de9075232854ebca6189aa90d928a11: No bootstrap required, opened a new log
I20260812 02:38:34.107156 22126 ts_tablet_manager.cc:1397] T 484ed45ef3674c4ca25cf7c625d69e1c P 2de9075232854ebca6189aa90d928a11: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260812 02:38:34.107942 22125 tablet_bootstrap.cc:492] T 484ed45ef3674c4ca25cf7c625d69e1c P 9eaa55ad48b3402480437fe9868a38cd: No bootstrap required, opened a new log
I20260812 02:38:34.108206 22125 ts_tablet_manager.cc:1397] T 484ed45ef3674c4ca25cf7c625d69e1c P 9eaa55ad48b3402480437fe9868a38cd: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.000s
I20260812 02:38:34.115674 22125 raft_consensus.cc:348] T 484ed45ef3674c4ca25cf7c625d69e1c P 9eaa55ad48b3402480437fe9868a38cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9eaa55ad48b3402480437fe9868a38cd" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 43585 } } peers { permanent_uuid: "414e120080594464900790a1e1c6b621" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 40599 } } peers { permanent_uuid: "2de9075232854ebca6189aa90d928a11" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 33885 } }
I20260812 02:38:34.116333 22125 raft_consensus.cc:374] T 484ed45ef3674c4ca25cf7c625d69e1c P 9eaa55ad48b3402480437fe9868a38cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:38:34.116515 22125 raft_consensus.cc:729] T 484ed45ef3674c4ca25cf7c625d69e1c P 9eaa55ad48b3402480437fe9868a38cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9eaa55ad48b3402480437fe9868a38cd, State: Initialized, Role: FOLLOWER
I20260812 02:38:34.116257 22124 raft_consensus.cc:348] T 484ed45ef3674c4ca25cf7c625d69e1c P 414e120080594464900790a1e1c6b621 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9eaa55ad48b3402480437fe9868a38cd" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 43585 } } peers { permanent_uuid: "414e120080594464900790a1e1c6b621" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 40599 } } peers { permanent_uuid: "2de9075232854ebca6189aa90d928a11" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 33885 } }
I20260812 02:38:34.116825 22124 raft_consensus.cc:374] T 484ed45ef3674c4ca25cf7c625d69e1c P 414e120080594464900790a1e1c6b621 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:38:34.116964 22124 raft_consensus.cc:729] T 484ed45ef3674c4ca25cf7c625d69e1c P 414e120080594464900790a1e1c6b621 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 414e120080594464900790a1e1c6b621, State: Initialized, Role: FOLLOWER
I20260812 02:38:34.117383 22125 consensus_queue.cc:260] T 484ed45ef3674c4ca25cf7c625d69e1c P 9eaa55ad48b3402480437fe9868a38cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9eaa55ad48b3402480437fe9868a38cd" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 43585 } } peers { permanent_uuid: "414e120080594464900790a1e1c6b621" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 40599 } } peers { permanent_uuid: "2de9075232854ebca6189aa90d928a11" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 33885 } }
I20260812 02:38:34.117694 22124 consensus_queue.cc:260] T 484ed45ef3674c4ca25cf7c625d69e1c P 414e120080594464900790a1e1c6b621 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9eaa55ad48b3402480437fe9868a38cd" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 43585 } } peers { permanent_uuid: "414e120080594464900790a1e1c6b621" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 40599 } } peers { permanent_uuid: "2de9075232854ebca6189aa90d928a11" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 33885 } }
I20260812 02:38:34.118223 22126 raft_consensus.cc:348] T 484ed45ef3674c4ca25cf7c625d69e1c P 2de9075232854ebca6189aa90d928a11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9eaa55ad48b3402480437fe9868a38cd" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 43585 } } peers { permanent_uuid: "414e120080594464900790a1e1c6b621" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 40599 } } peers { permanent_uuid: "2de9075232854ebca6189aa90d928a11" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 33885 } }
I20260812 02:38:34.118858 22126 raft_consensus.cc:374] T 484ed45ef3674c4ca25cf7c625d69e1c P 2de9075232854ebca6189aa90d928a11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:38:34.119029 22126 raft_consensus.cc:729] T 484ed45ef3674c4ca25cf7c625d69e1c P 2de9075232854ebca6189aa90d928a11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2de9075232854ebca6189aa90d928a11, State: Initialized, Role: FOLLOWER
I20260812 02:38:34.119437 22125 ts_tablet_manager.cc:1428] T 484ed45ef3674c4ca25cf7c625d69e1c P 9eaa55ad48b3402480437fe9868a38cd: Time spent starting tablet: real 0.011s user 0.011s sys 0.000s
I20260812 02:38:34.119899 22126 consensus_queue.cc:260] T 484ed45ef3674c4ca25cf7c625d69e1c P 2de9075232854ebca6189aa90d928a11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9eaa55ad48b3402480437fe9868a38cd" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 43585 } } peers { permanent_uuid: "414e120080594464900790a1e1c6b621" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 40599 } } peers { permanent_uuid: "2de9075232854ebca6189aa90d928a11" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 33885 } }
I20260812 02:38:34.120553 22100 heartbeater.cc:503] Master 127.7.132.190:45229 was elected leader, sending a full tablet report...
I20260812 02:38:34.121346 22124 ts_tablet_manager.cc:1428] T 484ed45ef3674c4ca25cf7c625d69e1c P 414e120080594464900790a1e1c6b621: Time spent starting tablet: real 0.016s user 0.015s sys 0.000s
I20260812 02:38:34.122578 21934 heartbeater.cc:503] Master 127.7.132.190:45229 was elected leader, sending a full tablet report...
I20260812 02:38:34.124182 22126 ts_tablet_manager.cc:1428] T 484ed45ef3674c4ca25cf7c625d69e1c P 2de9075232854ebca6189aa90d928a11: Time spent starting tablet: real 0.017s user 0.016s sys 0.000s
I20260812 02:38:34.131479 22131 raft_consensus.cc:482] T 484ed45ef3674c4ca25cf7c625d69e1c P 414e120080594464900790a1e1c6b621 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 02:38:34.131771 22131 raft_consensus.cc:504] T 484ed45ef3674c4ca25cf7c625d69e1c P 414e120080594464900790a1e1c6b621 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9eaa55ad48b3402480437fe9868a38cd" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 43585 } } peers { permanent_uuid: "414e120080594464900790a1e1c6b621" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 40599 } } peers { permanent_uuid: "2de9075232854ebca6189aa90d928a11" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 33885 } }
I20260812 02:38:34.133680 22131 leader_election.cc:288] T 484ed45ef3674c4ca25cf7c625d69e1c P 414e120080594464900790a1e1c6b621 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9eaa55ad48b3402480437fe9868a38cd (127.7.132.129:43585), 2de9075232854ebca6189aa90d928a11 (127.7.132.130:33885)
I20260812 02:38:34.139838 22104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:38:34.133883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:43585 (local address 127.7.132.131:48639)
0812 02:38:34.134095 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0812 02:38:34.134118 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0812 02:38:34.134241 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:38:34.134609 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:38:34.134614 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:38:34.134642 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:38:34.134875 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:38:34.134890 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:38:34.136175 (+ 1285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:38:34.136179 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:38:34.137959 (+ 1780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:38:34.137976 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:38:34.139289 (+ 1313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:38:34.139293 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:38:34.139420 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:38:34.139445 (+ 25us) client_negotiation.cc:769] Sending connection context
0812 02:38:34.139532 (+ 87us) client_negotiation.cc:240] Negotiation successful
0812 02:38:34.139636 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56,"mutex_wait_us":3}
I20260812 02:38:34.139967 22120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:38:34.133980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:48639 (local address 127.7.132.129:43585)
0812 02:38:34.134205 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0812 02:38:34.134211 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:38:34.134233 (+ 22us) server_negotiation.cc:407] Connection header received
0812 02:38:34.134391 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:38:34.134397 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:38:34.134473 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:38:34.134593 (+ 120us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:38:34.135069 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:38:34.136021 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:38:34.138190 (+ 2169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:38:34.139135 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:38:34.139295 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:38:34.139371 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 02:38:34.139579 (+ 208us) server_negotiation.cc:299] Negotiation successful
0812 02:38:34.139719 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260812 02:38:34.140875 22121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:38:34.134705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:59295 (local address 127.7.132.130:33885)
0812 02:38:34.134925 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0812 02:38:34.134933 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:38:34.135366 (+ 433us) server_negotiation.cc:407] Connection header received
0812 02:38:34.135478 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:38:34.135483 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:38:34.135572 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:38:34.135665 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:38:34.136133 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:38:34.136829 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:38:34.138879 (+ 2050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:38:34.140073 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:38:34.140239 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:38:34.140316 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 02:38:34.140587 (+ 271us) server_negotiation.cc:299] Negotiation successful
0812 02:38:34.140686 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260812 02:38:34.141055 22133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:38:34.134327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.130:33885 (local address 127.7.132.131:59295)
0812 02:38:34.135268 (+ 941us) negotiation.cc:107] Waiting for socket to connect
0812 02:38:34.135283 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 02:38:34.135382 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:38:34.135676 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:38:34.135679 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:38:34.135698 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:38:34.135984 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:38:34.135991 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:38:34.136988 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:38:34.136994 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:38:34.138721 (+ 1727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:38:34.138734 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:38:34.140257 (+ 1523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:38:34.140263 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:38:34.140421 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:38:34.140450 (+ 29us) client_negotiation.cc:769] Sending connection context
0812 02:38:34.140558 (+ 108us) client_negotiation.cc:240] Negotiation successful
0812 02:38:34.140705 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":838,"thread_start_us":665,"threads_started":1}
I20260812 02:38:34.141119 21713 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "484ed45ef3674c4ca25cf7c625d69e1c" candidate_uuid: "414e120080594464900790a1e1c6b621" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9eaa55ad48b3402480437fe9868a38cd" is_pre_election: true
I20260812 02:38:34.141749 21713 raft_consensus.cc:2454] T 484ed45ef3674c4ca25cf7c625d69e1c P 9eaa55ad48b3402480437fe9868a38cd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 414e120080594464900790a1e1c6b621 in term 0.
I20260812 02:38:34.142114 21885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "484ed45ef3674c4ca25cf7c625d69e1c" candidate_uuid: "414e120080594464900790a1e1c6b621" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2de9075232854ebca6189aa90d928a11" is_pre_election: true
I20260812 02:38:34.142510 21962 leader_election.cc:302] T 484ed45ef3674c4ca25cf7c625d69e1c P 414e120080594464900790a1e1c6b621 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 414e120080594464900790a1e1c6b621, 9eaa55ad48b3402480437fe9868a38cd; no voters:
I20260812 02:38:34.142692 21885 raft_consensus.cc:2454] T 484ed45ef3674c4ca25cf7c625d69e1c P 2de9075232854ebca6189aa90d928a11 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 414e120080594464900790a1e1c6b621 in term 0.
I20260812 02:38:34.142920 22131 raft_consensus.cc:2781] T 484ed45ef3674c4ca25cf7c625d69e1c P 414e120080594464900790a1e1c6b621 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 02:38:34.143093 22131 raft_consensus.cc:482] T 484ed45ef3674c4ca25cf7c625d69e1c P 414e120080594464900790a1e1c6b621 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 02:38:34.143234 22131 raft_consensus.cc:3037] T 484ed45ef3674c4ca25cf7c625d69e1c P 414e120080594464900790a1e1c6b621 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:38:34.145566 22131 raft_consensus.cc:504] T 484ed45ef3674c4ca25cf7c625d69e1c P 414e120080594464900790a1e1c6b621 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9eaa55ad48b3402480437fe9868a38cd" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 43585 } } peers { permanent_uuid: "414e120080594464900790a1e1c6b621" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 40599 } } peers { permanent_uuid: "2de9075232854ebca6189aa90d928a11" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 33885 } }
I20260812 02:38:34.146291 22131 leader_election.cc:288] T 484ed45ef3674c4ca25cf7c625d69e1c P 414e120080594464900790a1e1c6b621 [CANDIDATE]: Term 1 election: Requested vote from peers 9eaa55ad48b3402480437fe9868a38cd (127.7.132.129:43585), 2de9075232854ebca6189aa90d928a11 (127.7.132.130:33885)
W20260812 02:38:34.146489 21935 tablet.cc:2369] T 484ed45ef3674c4ca25cf7c625d69e1c P 2de9075232854ebca6189aa90d928a11: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 02:38:34.146711 21713 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "484ed45ef3674c4ca25cf7c625d69e1c" candidate_uuid: "414e120080594464900790a1e1c6b621" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9eaa55ad48b3402480437fe9868a38cd"
I20260812 02:38:34.146801 21885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "484ed45ef3674c4ca25cf7c625d69e1c" candidate_uuid: "414e120080594464900790a1e1c6b621" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2de9075232854ebca6189aa90d928a11"
I20260812 02:38:34.146965 21713 raft_consensus.cc:3037] T 484ed45ef3674c4ca25cf7c625d69e1c P 9eaa55ad48b3402480437fe9868a38cd [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:38:34.147033 21885 raft_consensus.cc:3037] T 484ed45ef3674c4ca25cf7c625d69e1c P 2de9075232854ebca6189aa90d928a11 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:38:34.150416 21885 raft_consensus.cc:2454] T 484ed45ef3674c4ca25cf7c625d69e1c P 2de9075232854ebca6189aa90d928a11 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 414e120080594464900790a1e1c6b621 in term 1.
I20260812 02:38:34.150416 21713 raft_consensus.cc:2454] T 484ed45ef3674c4ca25cf7c625d69e1c P 9eaa55ad48b3402480437fe9868a38cd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 414e120080594464900790a1e1c6b621 in term 1.
I20260812 02:38:34.150892 21962 leader_election.cc:302] T 484ed45ef3674c4ca25cf7c625d69e1c P 414e120080594464900790a1e1c6b621 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 414e120080594464900790a1e1c6b621, 9eaa55ad48b3402480437fe9868a38cd; no voters:
I20260812 02:38:34.151182 22131 raft_consensus.cc:2781] T 484ed45ef3674c4ca25cf7c625d69e1c P 414e120080594464900790a1e1c6b621 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:38:34.151705 22131 raft_consensus.cc:686] T 484ed45ef3674c4ca25cf7c625d69e1c P 414e120080594464900790a1e1c6b621 [term 1 LEADER]: Becoming Leader. State: Replica: 414e120080594464900790a1e1c6b621, State: Running, Role: LEADER
I20260812 02:38:34.152281 22131 consensus_queue.cc:237] T 484ed45ef3674c4ca25cf7c625d69e1c P 414e120080594464900790a1e1c6b621 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9eaa55ad48b3402480437fe9868a38cd" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 43585 } } peers { permanent_uuid: "414e120080594464900790a1e1c6b621" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 40599 } } peers { permanent_uuid: "2de9075232854ebca6189aa90d928a11" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 33885 } }
I20260812 02:38:34.157351 21512 catalog_manager.cc:5495] T 484ed45ef3674c4ca25cf7c625d69e1c P 414e120080594464900790a1e1c6b621 reported cstate change: term changed from 0 to 1, leader changed from <none> to 414e120080594464900790a1e1c6b621 (127.7.132.131). New cstate: current_term: 1 leader_uuid: "414e120080594464900790a1e1c6b621" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9eaa55ad48b3402480437fe9868a38cd" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 43585 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "414e120080594464900790a1e1c6b621" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 40599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2de9075232854ebca6189aa90d928a11" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 33885 } health_report { overall_health: UNKNOWN } } }
I20260812 02:38:34.190753 22119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:38:34.185083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40126 (local address 127.7.132.131:40599)
0812 02:38:34.185285 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0812 02:38:34.185290 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:38:34.185309 (+ 19us) server_negotiation.cc:407] Connection header received
0812 02:38:34.185402 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:38:34.185406 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:38:34.185563 (+ 157us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:38:34.185739 (+ 176us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 02:38:34.186221 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:38:34.187048 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:38:34.188462 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:38:34.188858 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:38:34.188956 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:38:34.189590 (+ 634us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 02:38:34.190208 (+ 618us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 02:38:34.190329 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0812 02:38:34.190474 (+ 145us) server_negotiation.cc:299] Negotiation successful
0812 02:38:34.190548 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260812 02:38:34.199893 21713 raft_consensus.cc:1261] T 484ed45ef3674c4ca25cf7c625d69e1c P 9eaa55ad48b3402480437fe9868a38cd [term 1 FOLLOWER]: Refusing update from remote peer 414e120080594464900790a1e1c6b621: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:38:34.199894 21885 raft_consensus.cc:1261] T 484ed45ef3674c4ca25cf7c625d69e1c P 2de9075232854ebca6189aa90d928a11 [term 1 FOLLOWER]: Refusing update from remote peer 414e120080594464900790a1e1c6b621: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:38:34.200975 22134 consensus_queue.cc:1035] T 484ed45ef3674c4ca25cf7c625d69e1c P 414e120080594464900790a1e1c6b621 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2de9075232854ebca6189aa90d928a11" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 33885 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:38:34.201412 22131 consensus_queue.cc:1035] T 484ed45ef3674c4ca25cf7c625d69e1c P 414e120080594464900790a1e1c6b621 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9eaa55ad48b3402480437fe9868a38cd" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 43585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:38:34.218938 22139 mvcc.cc:205] Tried to move back new op lower bound from 7317513478950891520 to 7317513478772703232. Current Snapshot: MvccSnapshot[applied={T|T < 7317513478950891520}]
I20260812 02:38:34.228626 21513 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34682:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 02:38:34.228947 21513 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34682:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 02:38:34.232952 21513 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 166e5ea99c77472ca0c87ae3b059e9c7: Sending DeleteTablet for 3 replicas of tablet 484ed45ef3674c4ca25cf7c625d69e1c
I20260812 02:38:34.234292 21690 tablet_service.cc:1514] Processing DeleteTablet for tablet 484ed45ef3674c4ca25cf7c625d69e1c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:38:34 UTC) from {username='kudu', principal='kudu/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:54216
I20260812 02:38:34.234316 22020 tablet_service.cc:1514] Processing DeleteTablet for tablet 484ed45ef3674c4ca25cf7c625d69e1c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:38:34 UTC) from {username='kudu', principal='kudu/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:40112
I20260812 02:38:34.235289 22149 tablet_replica.cc:330] T 484ed45ef3674c4ca25cf7c625d69e1c P 9eaa55ad48b3402480437fe9868a38cd: stopping tablet replica
I20260812 02:38:34.235785 22149 raft_consensus.cc:2229] T 484ed45ef3674c4ca25cf7c625d69e1c P 9eaa55ad48b3402480437fe9868a38cd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 02:38:34.235821 21859 tablet_service.cc:1514] Processing DeleteTablet for tablet 484ed45ef3674c4ca25cf7c625d69e1c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:38:34 UTC) from {username='kudu', principal='kudu/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:48818
I20260812 02:38:34.236339 22149 raft_consensus.cc:2258] T 484ed45ef3674c4ca25cf7c625d69e1c P 9eaa55ad48b3402480437fe9868a38cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:38:34.237210 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 21609
I20260812 02:38:34.237943 22147 tablet_replica.cc:330] T 484ed45ef3674c4ca25cf7c625d69e1c P 414e120080594464900790a1e1c6b621: stopping tablet replica
I20260812 02:38:34.240113 22149 ts_tablet_manager.cc:1905] T 484ed45ef3674c4ca25cf7c625d69e1c P 9eaa55ad48b3402480437fe9868a38cd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:38:34.245913 22151 tablet_replica.cc:330] T 484ed45ef3674c4ca25cf7c625d69e1c P 2de9075232854ebca6189aa90d928a11: stopping tablet replica
I20260812 02:38:34.246624 22151 raft_consensus.cc:2229] T 484ed45ef3674c4ca25cf7c625d69e1c P 2de9075232854ebca6189aa90d928a11 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 02:38:34.247259 22151 raft_consensus.cc:2258] T 484ed45ef3674c4ca25cf7c625d69e1c P 2de9075232854ebca6189aa90d928a11 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:38:34.248030 22147 raft_consensus.cc:2229] T 484ed45ef3674c4ca25cf7c625d69e1c P 414e120080594464900790a1e1c6b621 [term 1 LEADER]: Raft consensus shutting down.
I20260812 02:38:34.249563 22149 ts_tablet_manager.cc:1918] T 484ed45ef3674c4ca25cf7c625d69e1c P 9eaa55ad48b3402480437fe9868a38cd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 02:38:34.249826 22149 log.cc:1196] T 484ed45ef3674c4ca25cf7c625d69e1c P 9eaa55ad48b3402480437fe9868a38cd: Deleting WAL directory at /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/minicluster-data/ts-0/wal/wals/484ed45ef3674c4ca25cf7c625d69e1c
I20260812 02:38:34.250154 22151 ts_tablet_manager.cc:1905] T 484ed45ef3674c4ca25cf7c625d69e1c P 2de9075232854ebca6189aa90d928a11: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:38:34.250679 22149 ts_tablet_manager.cc:1939] T 484ed45ef3674c4ca25cf7c625d69e1c P 9eaa55ad48b3402480437fe9868a38cd: Deleting consensus metadata
I20260812 02:38:34.251029 22147 raft_consensus.cc:2258] T 484ed45ef3674c4ca25cf7c625d69e1c P 414e120080594464900790a1e1c6b621 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:38:34.252810 21497 catalog_manager.cc:4841] TS 9eaa55ad48b3402480437fe9868a38cd (127.7.132.129:43585): tablet 484ed45ef3674c4ca25cf7c625d69e1c (table test-table [id=4dede43767a548f8a0ad766947a5f587]) successfully deleted
I20260812 02:38:34.258698 22151 ts_tablet_manager.cc:1918] T 484ed45ef3674c4ca25cf7c625d69e1c P 2de9075232854ebca6189aa90d928a11: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
Aug 12 02:38:34 dist-test-slave-mn4r krb5kdc[20955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502309, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.132.129@KRBTEST.COM
I20260812 02:38:34.260777 22147 ts_tablet_manager.cc:1905] T 484ed45ef3674c4ca25cf7c625d69e1c P 414e120080594464900790a1e1c6b621: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:38:34.268056 22120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:38:34.238777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54230 (local address 127.7.132.129:43585)
0812 02:38:34.238966 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0812 02:38:34.238972 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:38:34.248731 (+ 9759us) server_negotiation.cc:407] Connection header received
0812 02:38:34.249239 (+ 508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:38:34.249246 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:38:34.249336 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:38:34.249419 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:38:34.251769 (+ 2350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:38:34.252874 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:38:34.254695 (+ 1821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:38:34.255274 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:38:34.255418 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:38:34.260917 (+ 5499us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:38:34.260953 (+ 36us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:38:34.260973 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:38:34.261020 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:38:34.264555 (+ 3535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:38:34.265500 (+ 945us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:38:34.265513 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:38:34.265515 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:38:34.265695 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:38:34.266148 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:38:34.266153 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:38:34.266156 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:38:34.266585 (+ 429us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:38:34.266930 (+ 345us) server_negotiation.cc:1035] Waiting for connection context
0812 02:38:34.267248 (+ 318us) server_negotiation.cc:299] Negotiation successful
0812 02:38:34.267512 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 02:38:34.268810 21760 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:38:34.269217 22147 ts_tablet_manager.cc:1918] T 484ed45ef3674c4ca25cf7c625d69e1c P 414e120080594464900790a1e1c6b621: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 02:38:34.259061 22151 log.cc:1196] T 484ed45ef3674c4ca25cf7c625d69e1c P 2de9075232854ebca6189aa90d928a11: Deleting WAL directory at /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/minicluster-data/ts-1/wal/wals/484ed45ef3674c4ca25cf7c625d69e1c
I20260812 02:38:34.270203 22151 ts_tablet_manager.cc:1939] T 484ed45ef3674c4ca25cf7c625d69e1c P 2de9075232854ebca6189aa90d928a11: Deleting consensus metadata
I20260812 02:38:34.273694 22147 log.cc:1196] T 484ed45ef3674c4ca25cf7c625d69e1c P 414e120080594464900790a1e1c6b621: Deleting WAL directory at /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786502156971023-7698-0/minicluster-data/ts-2/wal/wals/484ed45ef3674c4ca25cf7c625d69e1c
I20260812 02:38:34.273730 21497 catalog_manager.cc:4841] TS 2de9075232854ebca6189aa90d928a11 (127.7.132.130:33885): tablet 484ed45ef3674c4ca25cf7c625d69e1c (table test-table [id=4dede43767a548f8a0ad766947a5f587]) successfully deleted
I20260812 02:38:34.274487 22147 ts_tablet_manager.cc:1939] T 484ed45ef3674c4ca25cf7c625d69e1c P 414e120080594464900790a1e1c6b621: Deleting consensus metadata
I20260812 02:38:34.276710 21495 catalog_manager.cc:4841] TS 414e120080594464900790a1e1c6b621 (127.7.132.131:40599): tablet 484ed45ef3674c4ca25cf7c625d69e1c (table test-table [id=4dede43767a548f8a0ad766947a5f587]) successfully deleted
I20260812 02:38:34.363252 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 21609
I20260812 02:38:34.387140 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 21774
Aug 12 02:38:34 dist-test-slave-mn4r krb5kdc[20955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502309, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.132.130@KRBTEST.COM
I20260812 02:38:34.407050 22121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:38:34.388046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48826 (local address 127.7.132.130:33885)
0812 02:38:34.388231 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0812 02:38:34.388239 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:38:34.388319 (+ 80us) server_negotiation.cc:407] Connection header received
0812 02:38:34.388549 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:38:34.388559 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:38:34.388637 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:38:34.388744 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:38:34.390049 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:38:34.390821 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:38:34.392405 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:38:34.393128 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:38:34.393268 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:38:34.399209 (+ 5941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:38:34.399223 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:38:34.399241 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:38:34.399293 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:38:34.403475 (+ 4182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:38:34.404396 (+ 921us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:38:34.404407 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:38:34.404411 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:38:34.404577 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:38:34.405027 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:38:34.405033 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:38:34.405036 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:38:34.405502 (+ 466us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:38:34.405782 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0812 02:38:34.406386 (+ 604us) server_negotiation.cc:299] Negotiation successful
0812 02:38:34.406638 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260812 02:38:34.407703 21929 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:38:34.501583 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 21774
I20260812 02:38:34.522595 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 21943
Aug 12 02:38:34 dist-test-slave-mn4r krb5kdc[20955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502309, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.132.131@KRBTEST.COM
I20260812 02:38:34.541921 22119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:38:34.523669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40130 (local address 127.7.132.131:40599)
0812 02:38:34.523851 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0812 02:38:34.523855 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 02:38:34.523885 (+ 30us) server_negotiation.cc:407] Connection header received
0812 02:38:34.523950 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:38:34.523953 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:38:34.524045 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:38:34.524159 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:38:34.526185 (+ 2026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:38:34.527057 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:38:34.528539 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:38:34.529143 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:38:34.529287 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:38:34.534315 (+ 5028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:38:34.534334 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:38:34.534356 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:38:34.534439 (+ 83us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:38:34.538610 (+ 4171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:38:34.539449 (+ 839us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:38:34.539460 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:38:34.539465 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:38:34.539688 (+ 223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:38:34.540089 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:38:34.540095 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:38:34.540098 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:38:34.540622 (+ 524us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:38:34.540988 (+ 366us) server_negotiation.cc:1035] Waiting for connection context
0812 02:38:34.541159 (+ 171us) server_negotiation.cc:299] Negotiation successful
0812 02:38:34.541457 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260812 02:38:34.542747 22095 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:38:34.677320 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 21943
I20260812 02:38:34.699095 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 21477
I20260812 02:38:34.700397 21549 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:38:34.817662 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 21477
I20260812 02:38:34.845005 7698 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260812 02:38:35.191041 7698 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260812 02:38:35.191394 7698 mini_ranger.cc:66] Stopping Ranger...
I20260812 02:38:35.537216 7698 mini_ranger.cc:68] Stopped Ranger
2026-08-12 02:38:35.537 UTC [20988] LOG: received smart shutdown request
2026-08-12 02:38:35.541 UTC [20988] LOG: background worker "logical replication launcher" (PID 21002) exited with exit code 1
2026-08-12 02:38:35.542 UTC [20994] LOG: shutting down
2026-08-12 02:38:35.733 UTC [20988] LOG: database system is shut down
2026-08-12T02:38:35Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (54263 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 02:38:35 dist-test-slave-mn4r krb5kdc[22169](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 02:38:35 dist-test-slave-mn4r krb5kdc[22169](info): set up 2 sockets
Aug 12 02:38:35 dist-test-slave-mn4r krb5kdc[22169](info): commencing operation
krb5kdc: starting...
W20260812 02:38:37.143225 7698 mini_kdc.cc:121] Time spent starting KDC: real 1.234s user 0.000s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/krb5kdc/test-user.keytab.
Aug 12 02:38:37 dist-test-slave-mn4r krb5kdc[22169](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502317, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T02:38:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T02:38:37Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.7.132.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.7.132.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.7.132.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/krb5kdc/rangeradmin_127.7.132.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.7.132.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/krb5kdc/rangeradmin_127.7.132.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.7.132.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.7.132.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.7.132.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/krb5kdc/rangerlookup_127.7.132.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.7.132.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/krb5kdc/rangerlookup_127.7.132.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.7.132.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.7.132.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.7.132.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/krb5kdc/HTTP_127.7.132.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.7.132.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/krb5kdc/HTTP_127.7.132.148@KRBTEST.COM.keytab.
I20260812 02:38:37.551169 7698 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskSMpwqb/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/postgres -l logfile start
2026-08-12 02:38:40.153 UTC [22207] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-12 02:38:40.153 UTC [22207] LOG: listening on IPv4 address "127.7.132.148", port 33823
2026-08-12 02:38:40.157 UTC [22207] LOG: listening on Unix socket "/tmp/.s.PGSQL.33823"
2026-08-12 02:38:40.179 UTC [22210] LOG: database system was shut down at 2026-08-12 02:38:38 UTC
2026-08-12 02:38:40.183 UTC [22207] LOG: database system is ready to accept connections
I20260812 02:38:41.271683 7698 mini_postgres.cc:96] Postgres bound to 33823
2026-08-12 02:38:41.295 UTC [22218] FATAL: database "slave" does not exist
127.7.132.148:33823 - accepting connections
I20260812 02:38:41.295908 7698 mini_ranger.cc:161] Starting Ranger...
I20260812 02:38:41.327987 7698 mini_ranger.cc:84] Created miniranger Postgres user
I20260812 02:38:41.847605 7698 mini_ranger.cc:87] Created ranger Postgres database
I20260812 02:38:41.847733 7698 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/ranger-admin
2026-08-12 02:38:42,152 [I] DB FLAVOR :POSTGRES
2026-08-12 02:38:42,152 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 02:38:42,152 [I] Checking connection..
2026-08-12 02:38:42,152 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSMpwqb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskSMpwqb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.132.148:33823/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-12 02:38:42,442 [I] Checking connection passed.
2026-08-12 02:38:42,442 [I] --------- Verifying version history table ---------
2026-08-12 02:38:42,442 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSMpwqb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskSMpwqb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.132.148:33823/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 02:38:42,718 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 02:38:42,718 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSMpwqb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskSMpwqb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.132.148:33823/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 02:38:42,978 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 02:38:42,979 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-12 02:38:42,979 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSMpwqb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskSMpwqb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.132.148:33823/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskSMpwqb/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-12 02:38:43.249 UTC [22294] WARNING: there is no transaction in progress
2026-08-12 02:38:43,258 [I] create_dbversion_catalog.sql file imported successfully
2026-08-12 02:38:43,258 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSMpwqb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskSMpwqb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.132.148:33823/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 02:38:43,510 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-12 02:38:43,510 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 02:38:43,510 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSMpwqb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskSMpwqb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.132.148:33823/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-12 02:38:43,775 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSMpwqb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskSMpwqb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.132.148:33823/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-12 02:38:44,017 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSMpwqb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskSMpwqb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.132.148:33823/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-mn4r.c.gcp-upstream.internal','N') ;"
2026-08-12 02:38:44,266 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSMpwqb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskSMpwqb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.132.148:33823/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 02:38:44,550 [I] Table x_portal_user does not exist in database ranger
2026-08-12 02:38:44,550 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSMpwqb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskSMpwqb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.132.148:33823/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 02:38:44,804 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-12 02:38:44,804 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSMpwqb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskSMpwqb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.132.148:33823/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 02:38:45,064 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSMpwqb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskSMpwqb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.132.148:33823/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 02:38:45,309 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-12 02:38:45,309 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSMpwqb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskSMpwqb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.132.148:33823/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskSMpwqb/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-12 02:38:46.299 UTC [22425] WARNING: there is no transaction in progress
2026-08-12 02:38:46.316 UTC [22425] WARNING: there is no transaction in progress
2026-08-12 02:38:46.333 UTC [22425] WARNING: there is no transaction in progress
2026-08-12 02:38:46.350 UTC [22425] WARNING: there is no transaction in progress
2026-08-12 02:38:46.368 UTC [22425] WARNING: there is no transaction in progress
2026-08-12 02:38:46.386 UTC [22425] WARNING: there is no transaction in progress
2026-08-12 02:38:46.451 UTC [22425] WARNING: there is no transaction in progress
2026-08-12 02:38:46.458 UTC [22425] WARNING: there is no transaction in progress
2026-08-12 02:38:46.473 UTC [22425] WARNING: there is no transaction in progress
2026-08-12 02:38:46.484 UTC [22425] WARNING: there is no transaction in progress
2026-08-12 02:38:46.494 UTC [22425] WARNING: there is no transaction in progress
2026-08-12 02:38:46.508 UTC [22425] WARNING: there is no transaction in progress
2026-08-12 02:38:46.518 UTC [22425] WARNING: there is no transaction in progress
2026-08-12 02:38:46.524 UTC [22425] WARNING: there is no transaction in progress
2026-08-12 02:38:46.530 UTC [22425] WARNING: there is no transaction in progress
2026-08-12 02:38:46.536 UTC [22425] WARNING: there is no transaction in progress
2026-08-12 02:38:47,197 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-12 02:38:47,197 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSMpwqb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskSMpwqb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.132.148:33823/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-12 02:38:47,464 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-12 02:38:47,464 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSMpwqb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskSMpwqb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.132.148:33823/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 02:38:47,728 [I] Table x_portal_user already exists in database 'ranger'
2026-08-12 02:38:47,728 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSMpwqb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskSMpwqb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.132.148:33823/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 02:38:47,995 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-12 02:38:47,995 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSMpwqb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskSMpwqb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.132.148:33823/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 02:38:48,257 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSMpwqb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskSMpwqb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.132.148:33823/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 02:38:48,534 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSMpwqb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskSMpwqb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.132.148:33823/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-mn4r.c.gcp-upstream.internal';"
2026-08-12 02:38:48,795 [I] CORE_DB_SCHEMA import status has been updated
2026-08-12 02:38:48,796 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSMpwqb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskSMpwqb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.132.148:33823/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-12 02:38:49,062 [I] DB_PATCHES have already been applied
I20260812 02:38:49.069134 7698 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/ranger-admin:/tmp/dist-test-taskSMpwqb/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskSMpwqb/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskSMpwqb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskSMpwqb/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260812 02:38:49.069267 7698 mini_ranger.cc:193] Using host: 127.7.132.148
I20260812 02:38:49.089887 7698 mini_ranger.cc:227] Ranger admin URL: http://127.7.132.148:37907
Aug 12, 2026 2:38:49 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 2:38:49 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 2:38:49 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/ranger-admin/ews/webapp
Aug 12, 2026 2:38:49 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 12, 2026 2:38:49 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 12, 2026 2:38:49 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/ranger-admin/ews/webapp].
Aug 12, 2026 2:38:49 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260812 02:38:50.365957 7698 mini_ranger.cc:160] Time spent starting Ranger: real 9.070s user 0.004s sys 0.056s
Aug 12, 2026 2:39:11 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 12, 2026 2:39:11 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.MetricsREST
Aug 12, 2026 2:39:11 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 12, 2026 2:39:11 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 12, 2026 2:39:11 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 12, 2026 2:39:11 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 12, 2026 2:39:11 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 12, 2026 2:39:11 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 12, 2026 2:39:11 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 12, 2026 2:39:11 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 12, 2026 2:39:11 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 12, 2026 2:39:11 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 12, 2026 2:39:11 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 12, 2026 2:39:11 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 12, 2026 2:39:11 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 12, 2026 2:39:11 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 12, 2026 2:39:11 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 12, 2026 2:39:12 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260812 02:39:12.903903 7698 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.7.132.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.7.132.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.7.132.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/krb5kdc/rangerkms_127.7.132.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.7.132.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/krb5kdc/rangerkms_127.7.132.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.7.132.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.7.132.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.7.132.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/krb5kdc/HTTP_127.7.132.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.7.132.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/krb5kdc/HTTP_127.7.132.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 12 02:39:13 dist-test-slave-mn4r krb5kdc[22169](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502353, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260812 02:39:13.137187 7698 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260812 02:39:13.163431 7698 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260812 02:39:14.058969 7698 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260812 02:39:14.059095 7698 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/ranger-kms
I20260812 02:39:14.059154 7698 mini_ranger_kms.cc:227] Using postgres at 127.7.132.148:33823
2026-08-12 02:39:14,438 [I] DB FLAVOR :POSTGRES
2026-08-12 02:39:14,439 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 02:39:14,439 [I] Checking connection
2026-08-12 02:39:14,439 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSMpwqb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.132.148:33823/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-12 02:39:14,705 [I] connection success
2026-08-12 02:39:14,705 [I] --------- Verifying Ranger DB tables ---------
2026-08-12 02:39:14,705 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-12 02:39:14,705 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSMpwqb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.132.148:33823/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-12 02:39:14,955 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-12 02:39:14,955 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 02:39:14,956 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-12 02:39:14,956 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSMpwqb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.132.148:33823/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-12 02:39:15,215 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260812 02:39:15.691438 7698 mini_ranger_kms.cc:310] Created kms service
I20260812 02:39:15.920132 7698 mini_ranger_kms.cc:326] Created kudu user
I20260812 02:39:15.975925 7698 mini_ranger_kms.cc:343] Created rangerkms user
Aug 12 02:39:15 dist-test-slave-mn4r krb5kdc[22169](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502353, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.7.132.148@KRBTEST.COM
I20260812 02:39:16.438979 7698 mini_ranger_kms.cc:384] Added ranger policy
I20260812 02:39:16.439347 7698 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/ranger-kms:/tmp/dist-test-taskSMpwqb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskSMpwqb/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskSMpwqb/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260812 02:39:16.439411 7698 mini_ranger_kms.cc:240] Using host: 127.7.132.148
I20260812 02:39:16.460532 7698 mini_ranger_kms.cc:276] Ranger KMS PID: 22650
I20260812 02:39:16.460678 7698 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.7.132.148:35801
Aug 12, 2026 2:39:17 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 2:39:17 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 2:39:17 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 12, 2026 2:39:17 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 12, 2026 2:39:18 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/ranger-kms/ews/webapp].
Aug 12, 2026 2:39:18 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 12 02:39:18 dist-test-slave-mn4r krb5kdc[22169](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786502358, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.132.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:39:19.024045 7698 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.887s user 0.078s sys 0.332s
I20260812 02:39:19.024323 7698 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260812 02:39:19.024530 7698 mini_ranger_kms.cc:401] 127.7.132.148:35801/kms/v1/keys
Aug 12 02:39:19 dist-test-slave-mn4r krb5kdc[22169](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502358, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.132.148@KRBTEST.COM for HTTP/127.7.132.148@KRBTEST.COM
Aug 12 02:39:19 dist-test-slave-mn4r krb5kdc[22169](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502358, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.132.148@KRBTEST.COM for HTTP/127.7.132.148@KRBTEST.COM
Aug 12 02:39:21 dist-test-slave-mn4r krb5kdc[22169](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502361, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.7.132.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
I20260812 02:39:22.050284 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:40973
--webserver_interface=127.7.132.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:41657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.132.190:40973
--ranger_config_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.132.148:35801/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:39:22.400810 22714 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:39:22.401144 22714 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:39:22.401322 22714 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:39:22.409782 22714 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:39:22.409955 22714 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:39:22.410048 22714 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:39:22.410161 22714 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:39:22.410243 22714 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:39:22.421875 22714 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:41657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/master-0/wal
--ranger_kms_url=127.7.132.148:35801/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.7.132.190:40973
--ranger_config_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:40973
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.132.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:39:22.423485 22714 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:39:22.425046 22714 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:39:22.439368 22720 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:39:22.439435 22721 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:39:22.439461 22723 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:39:22.440183 22714 server_base.cc:1034] running on GCE node
I20260812 02:39:22.441095 22714 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:39:22.442752 22714 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:39:22.444046 22714 hybrid_clock.cc:648] HybridClock initialized: now 1786502362443964 us; error 50 us; skew 500 ppm
I20260812 02:39:22.444451 22714 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:39:22 dist-test-slave-mn4r krb5kdc[22169](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502362, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:39:22.447705 22714 init.cc:377] Logged in from keytab as kudu/127.7.132.190@KRBTEST.COM (short username kudu)
I20260812 02:39:22.449251 22714 webserver.cc:460] Webserver started at http://127.7.132.190:32923/ using document root <none> and password file <none>
I20260812 02:39:22.449960 22714 fs_manager.cc:359] Metadata directory not provided
I20260812 02:39:22.450130 22714 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:39:22.450479 22714 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 02:39:22 dist-test-slave-mn4r krb5kdc[22169](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502362, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.190@KRBTEST.COM for HTTP/127.7.132.148@KRBTEST.COM
I20260812 02:39:22.501295 22714 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/master-0/data/instance:
uuid: "279e011032c94987b4f63d0ad2772a40"
format_stamp: "Formatted at 2026-08-12 02:39:22 on dist-test-slave-mn4r"
server_key: "a80a60b5a318f0cbb813b60b032762a5"
server_key_iv: "24584310b3a2685c1465d8f42ef3b3f4"
server_key_version: "kuduclusterkey@0"
I20260812 02:39:22.502575 22714 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/master-0/wal/instance:
uuid: "279e011032c94987b4f63d0ad2772a40"
format_stamp: "Formatted at 2026-08-12 02:39:22 on dist-test-slave-mn4r"
server_key: "a80a60b5a318f0cbb813b60b032762a5"
server_key_iv: "24584310b3a2685c1465d8f42ef3b3f4"
server_key_version: "kuduclusterkey@0"
I20260812 02:39:22.510301 22714 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.001s
I20260812 02:39:22.531342 22733 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:39:22.533274 22714 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260812 02:39:22.533499 22714 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/master-0/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/master-0/wal
uuid: "279e011032c94987b4f63d0ad2772a40"
format_stamp: "Formatted at 2026-08-12 02:39:22 on dist-test-slave-mn4r"
server_key: "a80a60b5a318f0cbb813b60b032762a5"
server_key_iv: "24584310b3a2685c1465d8f42ef3b3f4"
server_key_version: "kuduclusterkey@0"
I20260812 02:39:22.533772 22714 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:39:22.563545 22714 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:39:22.567461 22714 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:39:22.567744 22714 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:39:22.586440 22714 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.190:40973
I20260812 02:39:22.586462 22791 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.190:40973 every 8 connection(s)
I20260812 02:39:22.587841 22714 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
I20260812 02:39:22.591528 22793 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:39:22.593519 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 22714
I20260812 02:39:22.593729 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/master-0/wal/instance
I20260812 02:39:22.594501 7698 external_mini_cluster.cc:1428] Setting key 82204a9f8932dae192399c21290d488f
I20260812 02:39:22.605115 22793 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 279e011032c94987b4f63d0ad2772a40: Bootstrap starting.
I20260812 02:39:22.608896 22793 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 279e011032c94987b4f63d0ad2772a40: Neither blocks nor log segments found. Creating new log.
I20260812 02:39:22.609900 22793 log.cc:824] T 00000000000000000000000000000000 P 279e011032c94987b4f63d0ad2772a40: Log is configured to *not* fsync() on all Append() calls
Aug 12 02:39:22 dist-test-slave-mn4r krb5kdc[22169](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502361, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 02:39:22.613747 22793 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 279e011032c94987b4f63d0ad2772a40: No bootstrap required, opened a new log
I20260812 02:39:22.619532 22796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:22.596719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57628 (local address 127.7.132.190:40973)
0812 02:39:22.597578 (+ 859us) server_negotiation.cc:206] Beginning negotiation
0812 02:39:22.597595 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 02:39:22.597650 (+ 55us) server_negotiation.cc:407] Connection header received
0812 02:39:22.600026 (+ 2376us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:39:22.600057 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:39:22.600570 (+ 513us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:39:22.601064 (+ 494us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:39:22.602374 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:22.604409 (+ 2035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:22.605875 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:22.606969 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:22.607184 (+ 215us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:39:22.611661 (+ 4477us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:39:22.611677 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:39:22.611698 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:39:22.611745 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:39:22.615238 (+ 3493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:39:22.615956 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:39:22.615975 (+ 19us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:39:22.615981 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:39:22.616155 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:39:22.616575 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:39:22.616580 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:39:22.616583 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:39:22.617149 (+ 566us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:39:22.617482 (+ 333us) server_negotiation.cc:1035] Waiting for connection context
0812 02:39:22.617992 (+ 510us) server_negotiation.cc:299] Negotiation successful
0812 02:39:22.618445 (+ 453us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":605,"thread_start_us":379,"threads_started":1}
I20260812 02:39:22.622452 22793 raft_consensus.cc:348] T 00000000000000000000000000000000 P 279e011032c94987b4f63d0ad2772a40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "279e011032c94987b4f63d0ad2772a40" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 40973 } }
I20260812 02:39:22.622956 22793 raft_consensus.cc:374] T 00000000000000000000000000000000 P 279e011032c94987b4f63d0ad2772a40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:39:22.623081 22793 raft_consensus.cc:729] T 00000000000000000000000000000000 P 279e011032c94987b4f63d0ad2772a40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 279e011032c94987b4f63d0ad2772a40, State: Initialized, Role: FOLLOWER
I20260812 02:39:22.623693 22793 consensus_queue.cc:260] T 00000000000000000000000000000000 P 279e011032c94987b4f63d0ad2772a40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "279e011032c94987b4f63d0ad2772a40" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 40973 } }
I20260812 02:39:22.623936 22793 raft_consensus.cc:388] T 00000000000000000000000000000000 P 279e011032c94987b4f63d0ad2772a40 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 02:39:22.624074 22793 raft_consensus.cc:482] T 00000000000000000000000000000000 P 279e011032c94987b4f63d0ad2772a40 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 02:39:22.624254 22793 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 279e011032c94987b4f63d0ad2772a40 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:39:22.626734 22793 raft_consensus.cc:504] T 00000000000000000000000000000000 P 279e011032c94987b4f63d0ad2772a40 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "279e011032c94987b4f63d0ad2772a40" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 40973 } }
I20260812 02:39:22.627252 22793 leader_election.cc:302] T 00000000000000000000000000000000 P 279e011032c94987b4f63d0ad2772a40 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 279e011032c94987b4f63d0ad2772a40; no voters:
I20260812 02:39:22.627827 22793 leader_election.cc:288] T 00000000000000000000000000000000 P 279e011032c94987b4f63d0ad2772a40 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 02:39:22.627961 22798 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 279e011032c94987b4f63d0ad2772a40 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:39:22.628680 22798 raft_consensus.cc:686] T 00000000000000000000000000000000 P 279e011032c94987b4f63d0ad2772a40 [term 1 LEADER]: Becoming Leader. State: Replica: 279e011032c94987b4f63d0ad2772a40, State: Running, Role: LEADER
I20260812 02:39:22.629227 22798 consensus_queue.cc:237] T 00000000000000000000000000000000 P 279e011032c94987b4f63d0ad2772a40 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "279e011032c94987b4f63d0ad2772a40" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 40973 } }
I20260812 02:39:22.629923 22793 sys_catalog.cc:564] T 00000000000000000000000000000000 P 279e011032c94987b4f63d0ad2772a40 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:39:22.632237 22800 sys_catalog.cc:455] T 00000000000000000000000000000000 P 279e011032c94987b4f63d0ad2772a40 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 279e011032c94987b4f63d0ad2772a40. Latest consensus state: current_term: 1 leader_uuid: "279e011032c94987b4f63d0ad2772a40" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "279e011032c94987b4f63d0ad2772a40" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 40973 } } }
I20260812 02:39:22.632618 22800 sys_catalog.cc:458] T 00000000000000000000000000000000 P 279e011032c94987b4f63d0ad2772a40 [sys.catalog]: This master's current role is: LEADER
I20260812 02:39:22.632534 22799 sys_catalog.cc:455] T 00000000000000000000000000000000 P 279e011032c94987b4f63d0ad2772a40 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "279e011032c94987b4f63d0ad2772a40" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "279e011032c94987b4f63d0ad2772a40" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 40973 } } }
I20260812 02:39:22.635084 22799 sys_catalog.cc:458] T 00000000000000000000000000000000 P 279e011032c94987b4f63d0ad2772a40 [sys.catalog]: This master's current role is: LEADER
I20260812 02:39:22.636193 22805 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:39:22.641170 22805 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 02:39:22.641983 22793 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260812 02:39:22.658628 22805 catalog_manager.cc:1347] Generated new cluster ID: ebf0307fb5c946d1a54fc9bc835a2779
I20260812 02:39:22.658787 22805 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 02:39:22.678964 22805 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 02:39:22.681031 22805 catalog_manager.cc:1504] Loading token signing keys...
I20260812 02:39:22.697728 22805 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 279e011032c94987b4f63d0ad2772a40: Generated new TSK 0
I20260812 02:39:22.699056 22805 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.132.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.129.keytab.
Entry for principal kudu/127.7.132.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.129.keytab.
I20260812 02:39:23.427273 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.132.148:35801/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.129:0
--local_ip_for_outbound_sockets=127.7.132.129
--webserver_interface=127.7.132.129
--webserver_port=0
--tserver_master_addrs=127.7.132.190:40973
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.129.keytab
--principal=kudu/127.7.132.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:41657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 02:39:23 dist-test-slave-mn4r krb5kdc[22169](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786502363, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260812 02:39:23.900772 22846 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:39:23.901204 22846 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:39:23.901427 22846 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:39:23.913173 22846 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:39:23.913316 22846 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:39:23.913569 22846 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.129
I20260812 02:39:23.929703 22846 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:41657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.7.132.148:35801/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.129.keytab
--principal=kudu/127.7.132.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.132.129
--webserver_port=0
--tserver_master_addrs=127.7.132.190:40973
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.129
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:39:23.931794 22846 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:39:23.933775 22846 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:39:23.951449 22859 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:39:23.953570 22860 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:39:23.967769 22862 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:39:23.970568 22846 server_base.cc:1034] running on GCE node
I20260812 02:39:23.971326 22846 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:39:23.972489 22846 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:39:23.973794 22846 hybrid_clock.cc:648] HybridClock initialized: now 1786502363973705 us; error 87 us; skew 500 ppm
I20260812 02:39:23.974217 22846 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:39:23 dist-test-slave-mn4r krb5kdc[22169](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502363, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:39:23.981851 22846 init.cc:377] Logged in from keytab as kudu/127.7.132.129@KRBTEST.COM (short username kudu)
I20260812 02:39:23.984319 22846 webserver.cc:460] Webserver started at http://127.7.132.129:37567/ using document root <none> and password file <none>
I20260812 02:39:23.985123 22846 fs_manager.cc:359] Metadata directory not provided
I20260812 02:39:23.985284 22846 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:39:23.985637 22846 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 02:39:23 dist-test-slave-mn4r krb5kdc[22169](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502363, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.129@KRBTEST.COM for HTTP/127.7.132.148@KRBTEST.COM
I20260812 02:39:24.009569 22846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/ts-0/data/instance:
uuid: "7c97d93336a7454db0e8720dfd306be0"
format_stamp: "Formatted at 2026-08-12 02:39:23 on dist-test-slave-mn4r"
server_key: "fefdbdc6d1f90353d5feb85b8d470214"
server_key_iv: "9cc9e88ea5d1ac5bcf5dae3388a5c204"
server_key_version: "kuduclusterkey@0"
I20260812 02:39:24.010640 22846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/ts-0/wal/instance:
uuid: "7c97d93336a7454db0e8720dfd306be0"
format_stamp: "Formatted at 2026-08-12 02:39:23 on dist-test-slave-mn4r"
server_key: "fefdbdc6d1f90353d5feb85b8d470214"
server_key_iv: "9cc9e88ea5d1ac5bcf5dae3388a5c204"
server_key_version: "kuduclusterkey@0"
I20260812 02:39:24.019414 22846 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.005s sys 0.003s
I20260812 02:39:24.041920 22871 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:39:24.044159 22846 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.000s sys 0.004s
I20260812 02:39:24.044423 22846 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/ts-0/wal
uuid: "7c97d93336a7454db0e8720dfd306be0"
format_stamp: "Formatted at 2026-08-12 02:39:23 on dist-test-slave-mn4r"
server_key: "fefdbdc6d1f90353d5feb85b8d470214"
server_key_iv: "9cc9e88ea5d1ac5bcf5dae3388a5c204"
server_key_version: "kuduclusterkey@0"
I20260812 02:39:24.044776 22846 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:39:24.070995 22846 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:39:24.085794 22846 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:39:24.086305 22846 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:39:24.088431 22846 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:39:24.094293 22846 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:39:24.094465 22846 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:39:24.094695 22846 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:39:24.094784 22846 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:39:24.173478 22846 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.129:46179
I20260812 02:39:24.173496 23007 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.129:46179 every 8 connection(s)
I20260812 02:39:24.175522 22846 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/ts-0/data/info.pb
I20260812 02:39:24.183830 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 22846
I20260812 02:39:24.184099 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/ts-0/wal/instance
I20260812 02:39:24.185129 7698 external_mini_cluster.cc:1428] Setting key d4d797ecfbd32979ffd49271a76d283e
Aug 12 02:39:24 dist-test-slave-mn4r krb5kdc[22169](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502363, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.129@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 02:39:24.211197 23013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:24.179838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:44771 (local address 127.7.132.190:40973)
0812 02:39:24.184021 (+ 4183us) server_negotiation.cc:206] Beginning negotiation
0812 02:39:24.184029 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:39:24.189787 (+ 5758us) server_negotiation.cc:407] Connection header received
0812 02:39:24.189871 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:39:24.189878 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:39:24.189995 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:39:24.190130 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:39:24.192459 (+ 2329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:24.193689 (+ 1230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:24.196488 (+ 2799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:24.197237 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:24.197356 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:39:24.202532 (+ 5176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:39:24.202542 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:39:24.202550 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:39:24.202609 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:39:24.206778 (+ 4169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:39:24.207854 (+ 1076us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:39:24.207862 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:39:24.207869 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:39:24.208061 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:39:24.208688 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:39:24.208696 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:39:24.208702 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:39:24.209122 (+ 420us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:39:24.209263 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0812 02:39:24.210390 (+ 1127us) server_negotiation.cc:299] Negotiation successful
0812 02:39:24.210651 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4014,"thread_start_us":3890,"threads_started":1}
I20260812 02:39:24.214013 23015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:24.180502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:40973 (local address 127.7.132.129:44771)
0812 02:39:24.186748 (+ 6246us) negotiation.cc:107] Waiting for socket to connect
0812 02:39:24.186812 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0812 02:39:24.189312 (+ 2500us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:39:24.190621 (+ 1309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:39:24.190637 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:39:24.191342 (+ 705us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:39:24.192227 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:39:24.192257 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:39:24.193933 (+ 1676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:39:24.193941 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:39:24.196300 (+ 2359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:39:24.196319 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:39:24.197758 (+ 1439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:39:24.197768 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:39:24.198146 (+ 378us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:39:24.199033 (+ 887us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:39:24.199069 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:39:24.202247 (+ 3178us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:39:24.206999 (+ 4752us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:39:24.207012 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:39:24.207033 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:39:24.207704 (+ 671us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:39:24.208219 (+ 515us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:39:24.208226 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:39:24.208232 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:39:24.208542 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:39:24.209372 (+ 830us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:39:24.209383 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:39:24.209895 (+ 512us) client_negotiation.cc:769] Sending connection context
0812 02:39:24.211796 (+ 1901us) client_negotiation.cc:240] Negotiation successful
0812 02:39:24.212425 (+ 629us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5904,"thread_start_us":13463,"threads_started":1}
I20260812 02:39:24.216392 23009 heartbeater.cc:347] Connected to a master server at 127.7.132.190:40973
I20260812 02:39:24.216738 23009 heartbeater.cc:464] Registering TS with master...
I20260812 02:39:24.217717 23009 heartbeater.cc:511] Master 127.7.132.190:40973 requested a full tablet report, sending...
I20260812 02:39:24.220302 22755 ts_manager.cc:194] Registered new tserver with Master: 7c97d93336a7454db0e8720dfd306be0 (127.7.132.129:46179)
I20260812 02:39:24.222646 22755 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.129@KRBTEST.COM'} at 127.7.132.129:44771
WARNING: no policy specified for kudu/127.7.132.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.130.keytab.
Entry for principal kudu/127.7.132.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.130.keytab.
Aug 12 02:39:24 dist-test-slave-mn4r krb5kdc[22169](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502363, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.190@KRBTEST.COM for HTTP/127.7.132.148@KRBTEST.COM
I20260812 02:39:24.271792 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.132.148:35801/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.130:0
--local_ip_for_outbound_sockets=127.7.132.130
--webserver_interface=127.7.132.130
--webserver_port=0
--tserver_master_addrs=127.7.132.190:40973
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.130.keytab
--principal=kudu/127.7.132.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:41657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 02:39:24 dist-test-slave-mn4r krb5kdc[22169](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502363, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.190@KRBTEST.COM for HTTP/127.7.132.148@KRBTEST.COM
W20260812 02:39:24.788317 23018 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:39:24.788853 23018 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:39:24.789072 23018 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:39:24.800879 23018 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:39:24.801115 23018 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:39:24.801348 23018 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.130
I20260812 02:39:24.817652 23018 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:41657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.7.132.148:35801/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.130.keytab
--principal=kudu/127.7.132.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.132.130
--webserver_port=0
--tserver_master_addrs=127.7.132.190:40973
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.130
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:39:24.819482 23018 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:39:24.821393 23018 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:39:24.844776 23026 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:39:24.845517 23027 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:39:24.853385 23029 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:39:24.861778 23018 server_base.cc:1034] running on GCE node
I20260812 02:39:24.862906 23018 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:39:24.868294 23018 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:39:24.870244 23018 hybrid_clock.cc:648] HybridClock initialized: now 1786502364870160 us; error 67 us; skew 500 ppm
I20260812 02:39:24.870754 23018 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:39:24 dist-test-slave-mn4r krb5kdc[22169](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502364, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:39:24.875504 23018 init.cc:377] Logged in from keytab as kudu/127.7.132.130@KRBTEST.COM (short username kudu)
I20260812 02:39:24.883651 23018 webserver.cc:460] Webserver started at http://127.7.132.130:45433/ using document root <none> and password file <none>
I20260812 02:39:24.884613 23018 fs_manager.cc:359] Metadata directory not provided
I20260812 02:39:24.884774 23018 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:39:24.885149 23018 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 02:39:24 dist-test-slave-mn4r krb5kdc[22169](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502364, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.130@KRBTEST.COM for HTTP/127.7.132.148@KRBTEST.COM
I20260812 02:39:24.911341 23018 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/ts-1/data/instance:
uuid: "20b9d93915064ca6b071511d633709cf"
format_stamp: "Formatted at 2026-08-12 02:39:24 on dist-test-slave-mn4r"
server_key: "1ede31db0ea993a311f74ef4a452f033"
server_key_iv: "c96d32683b301de116479f0f0937917e"
server_key_version: "kuduclusterkey@0"
I20260812 02:39:24.912676 23018 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/ts-1/wal/instance:
uuid: "20b9d93915064ca6b071511d633709cf"
format_stamp: "Formatted at 2026-08-12 02:39:24 on dist-test-slave-mn4r"
server_key: "1ede31db0ea993a311f74ef4a452f033"
server_key_iv: "c96d32683b301de116479f0f0937917e"
server_key_version: "kuduclusterkey@0"
I20260812 02:39:24.932283 23018 fs_manager.cc:691] Time spent creating directory manager: real 0.019s user 0.004s sys 0.012s
I20260812 02:39:24.976707 23038 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:39:24.983654 23018 fs_manager.cc:725] Time spent opening block manager: real 0.022s user 0.002s sys 0.013s
I20260812 02:39:24.983904 23018 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/ts-1/wal
uuid: "20b9d93915064ca6b071511d633709cf"
format_stamp: "Formatted at 2026-08-12 02:39:24 on dist-test-slave-mn4r"
server_key: "1ede31db0ea993a311f74ef4a452f033"
server_key_iv: "c96d32683b301de116479f0f0937917e"
server_key_version: "kuduclusterkey@0"
I20260812 02:39:24.984196 23018 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:39:25.026576 23018 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:39:25.032552 23018 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:39:25.032979 23018 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:39:25.034301 23018 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:39:25.036129 23018 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:39:25.036226 23018 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:39:25.036356 23018 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:39:25.036417 23018 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:39:25.079808 23018 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.130:37475
I20260812 02:39:25.079815 23180 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.130:37475 every 8 connection(s)
I20260812 02:39:25.081459 23018 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/ts-1/data/info.pb
I20260812 02:39:25.088207 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 23018
I20260812 02:39:25.088408 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/ts-1/wal/instance
I20260812 02:39:25.089032 7698 external_mini_cluster.cc:1428] Setting key 34f41bf12483b9893bdd64de8e78da19
Aug 12 02:39:25 dist-test-slave-mn4r krb5kdc[22169](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502364, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.130@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 02:39:25.108338 23183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:25.084269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:54985 (local address 127.7.132.190:40973)
0812 02:39:25.084822 (+ 553us) server_negotiation.cc:206] Beginning negotiation
0812 02:39:25.084828 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:39:25.086032 (+ 1204us) server_negotiation.cc:407] Connection header received
0812 02:39:25.088721 (+ 2689us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:39:25.088728 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:39:25.088843 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:39:25.088971 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:39:25.091022 (+ 2051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:25.092142 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:25.094862 (+ 2720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:25.095610 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:25.095768 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:39:25.100431 (+ 4663us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:39:25.100441 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:39:25.100449 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:39:25.100500 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:39:25.103998 (+ 3498us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:39:25.105097 (+ 1099us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:39:25.105106 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:39:25.105116 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:39:25.105282 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:39:25.105893 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:39:25.105901 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:39:25.105908 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:39:25.106374 (+ 466us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:39:25.106541 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0812 02:39:25.107489 (+ 948us) server_negotiation.cc:299] Negotiation successful
0812 02:39:25.107807 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":421,"thread_start_us":292,"threads_started":1}
I20260812 02:39:25.109431 23184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:25.084959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:40973 (local address 127.7.132.130:54985)
0812 02:39:25.085834 (+ 875us) negotiation.cc:107] Waiting for socket to connect
0812 02:39:25.085885 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 02:39:25.088357 (+ 2472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:39:25.089272 (+ 915us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:39:25.089285 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:39:25.089920 (+ 635us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:39:25.090826 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:39:25.090854 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:39:25.092347 (+ 1493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:39:25.092353 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:39:25.094685 (+ 2332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:39:25.094702 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:39:25.095890 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:39:25.095898 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:39:25.096221 (+ 323us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:39:25.097110 (+ 889us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:39:25.097142 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:39:25.100121 (+ 2979us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:39:25.104205 (+ 4084us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:39:25.104215 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:39:25.104236 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:39:25.104924 (+ 688us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:39:25.105424 (+ 500us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:39:25.105430 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:39:25.105433 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:39:25.105749 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:39:25.106534 (+ 785us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:39:25.106543 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:39:25.107020 (+ 477us) client_negotiation.cc:769] Sending connection context
0812 02:39:25.107340 (+ 320us) client_negotiation.cc:240] Negotiation successful
0812 02:39:25.107944 (+ 604us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":581,"thread_start_us":330,"threads_started":1}
I20260812 02:39:25.111330 23181 heartbeater.cc:347] Connected to a master server at 127.7.132.190:40973
I20260812 02:39:25.111675 23181 heartbeater.cc:464] Registering TS with master...
I20260812 02:39:25.112406 23181 heartbeater.cc:511] Master 127.7.132.190:40973 requested a full tablet report, sending...
I20260812 02:39:25.114039 22755 ts_manager.cc:194] Registered new tserver with Master: 20b9d93915064ca6b071511d633709cf (127.7.132.130:37475)
I20260812 02:39:25.115186 22755 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.130@KRBTEST.COM'} at 127.7.132.130:54985
WARNING: no policy specified for kudu/127.7.132.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.131.keytab.
Entry for principal kudu/127.7.132.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.131.keytab.
I20260812 02:39:25.157122 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.132.148:35801/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.131:0
--local_ip_for_outbound_sockets=127.7.132.131
--webserver_interface=127.7.132.131
--webserver_port=0
--tserver_master_addrs=127.7.132.190:40973
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.131.keytab
--principal=kudu/127.7.132.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:41657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 02:39:25.226688 23009 heartbeater.cc:503] Master 127.7.132.190:40973 was elected leader, sending a full tablet report...
W20260812 02:39:25.518648 23187 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:39:25.518986 23187 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:39:25.519165 23187 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:39:25.526607 23187 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:39:25.526749 23187 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:39:25.526927 23187 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.131
I20260812 02:39:25.537202 23187 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:41657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.7.132.148:35801/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.131.keytab
--principal=kudu/127.7.132.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.132.131
--webserver_port=0
--tserver_master_addrs=127.7.132.190:40973
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.131
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:39:25.538712 23187 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:39:25.540078 23187 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:39:25.554229 23195 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:39:25.554155 23197 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:39:25.554193 23193 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:39:25.555389 23187 server_base.cc:1034] running on GCE node
I20260812 02:39:25.556017 23187 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:39:25.556986 23187 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:39:25.558182 23187 hybrid_clock.cc:648] HybridClock initialized: now 1786502365558118 us; error 66 us; skew 500 ppm
I20260812 02:39:25.558509 23187 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:39:25 dist-test-slave-mn4r krb5kdc[22169](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502365, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:39:25.561738 23187 init.cc:377] Logged in from keytab as kudu/127.7.132.131@KRBTEST.COM (short username kudu)
I20260812 02:39:25.563470 23187 webserver.cc:460] Webserver started at http://127.7.132.131:40705/ using document root <none> and password file <none>
I20260812 02:39:25.564184 23187 fs_manager.cc:359] Metadata directory not provided
I20260812 02:39:25.564337 23187 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:39:25.564699 23187 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 02:39:25 dist-test-slave-mn4r krb5kdc[22169](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502365, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.131@KRBTEST.COM for HTTP/127.7.132.148@KRBTEST.COM
I20260812 02:39:25.579878 23187 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/ts-2/data/instance:
uuid: "4126d927aac74375b2d65068c2420061"
format_stamp: "Formatted at 2026-08-12 02:39:25 on dist-test-slave-mn4r"
server_key: "5efe139cc757fbaf8439fe8bf5c6f75f"
server_key_iv: "b22dbd3ade37610e147dfbbb783afb04"
server_key_version: "kuduclusterkey@0"
I20260812 02:39:25.580832 23187 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/ts-2/wal/instance:
uuid: "4126d927aac74375b2d65068c2420061"
format_stamp: "Formatted at 2026-08-12 02:39:25 on dist-test-slave-mn4r"
server_key: "5efe139cc757fbaf8439fe8bf5c6f75f"
server_key_iv: "b22dbd3ade37610e147dfbbb783afb04"
server_key_version: "kuduclusterkey@0"
I20260812 02:39:25.586755 23187 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260812 02:39:25.604130 23205 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:39:25.605748 23187 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260812 02:39:25.605952 23187 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/ts-2/wal
uuid: "4126d927aac74375b2d65068c2420061"
format_stamp: "Formatted at 2026-08-12 02:39:25 on dist-test-slave-mn4r"
server_key: "5efe139cc757fbaf8439fe8bf5c6f75f"
server_key_iv: "b22dbd3ade37610e147dfbbb783afb04"
server_key_version: "kuduclusterkey@0"
I20260812 02:39:25.606273 23187 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:39:25.650215 23187 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:39:25.654665 23187 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:39:25.654973 23187 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:39:25.655990 23187 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:39:25.657512 23187 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:39:25.657618 23187 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:39:25.657769 23187 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:39:25.657843 23187 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:39:25.691560 23187 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.131:41889
I20260812 02:39:25.691579 23334 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.131:41889 every 8 connection(s)
I20260812 02:39:25.692956 23187 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/ts-2/data/info.pb
I20260812 02:39:25.700140 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 23187
I20260812 02:39:25.700325 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/ts-2/wal/instance
I20260812 02:39:25.700971 7698 external_mini_cluster.cc:1428] Setting key 74d439b6ed7dd185ae13d4a1dfecdd75
Aug 12 02:39:25 dist-test-slave-mn4r krb5kdc[22169](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502365, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.131@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 02:39:25.714038 23337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:25.695508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:56025 (local address 127.7.132.190:40973)
0812 02:39:25.695975 (+ 467us) server_negotiation.cc:206] Beginning negotiation
0812 02:39:25.695980 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:39:25.697008 (+ 1028us) server_negotiation.cc:407] Connection header received
0812 02:39:25.698843 (+ 1835us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:39:25.698849 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:39:25.698952 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:39:25.699077 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:39:25.700909 (+ 1832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:25.701850 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:25.703733 (+ 1883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:25.704394 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:25.704539 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:39:25.708240 (+ 3701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:39:25.708246 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:39:25.708249 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:39:25.708285 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:39:25.711181 (+ 2896us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:39:25.711936 (+ 755us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:39:25.711940 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:39:25.711943 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:39:25.712080 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:39:25.712488 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:39:25.712492 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:39:25.712494 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:39:25.712768 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:39:25.712881 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0812 02:39:25.713592 (+ 711us) server_negotiation.cc:299] Negotiation successful
0812 02:39:25.713762 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":224,"threads_started":1}
I20260812 02:39:25.715488 23338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:25.696144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:40973 (local address 127.7.132.131:56025)
0812 02:39:25.696834 (+ 690us) negotiation.cc:107] Waiting for socket to connect
0812 02:39:25.696876 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 02:39:25.698568 (+ 1692us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:39:25.699316 (+ 748us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:39:25.699324 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:39:25.699821 (+ 497us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:39:25.700708 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:39:25.700736 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:39:25.702023 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:39:25.702030 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:39:25.703574 (+ 1544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:39:25.703587 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:39:25.704666 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:39:25.704674 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:39:25.704909 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:39:25.705649 (+ 740us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:39:25.705673 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:39:25.708003 (+ 2330us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:39:25.711318 (+ 3315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:39:25.711326 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:39:25.711340 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:39:25.711813 (+ 473us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:39:25.712188 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:39:25.712192 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:39:25.712194 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:39:25.712384 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:39:25.712877 (+ 493us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:39:25.712883 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:39:25.713238 (+ 355us) client_negotiation.cc:769] Sending connection context
0812 02:39:25.713492 (+ 254us) client_negotiation.cc:240] Negotiation successful
0812 02:39:25.713903 (+ 411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":276,"threads_started":1}
I20260812 02:39:25.717198 23335 heartbeater.cc:347] Connected to a master server at 127.7.132.190:40973
I20260812 02:39:25.717598 23335 heartbeater.cc:464] Registering TS with master...
I20260812 02:39:25.718444 23335 heartbeater.cc:511] Master 127.7.132.190:40973 requested a full tablet report, sending...
I20260812 02:39:25.719918 22755 ts_manager.cc:194] Registered new tserver with Master: 4126d927aac74375b2d65068c2420061 (127.7.132.131:41889)
I20260812 02:39:25.720794 22755 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.131@KRBTEST.COM'} at 127.7.132.131:56025
I20260812 02:39:25.730011 7698 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 02:39:25.732054 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 22846
Aug 12 02:39:25 dist-test-slave-mn4r krb5kdc[22169](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502361, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.132.129@KRBTEST.COM
I20260812 02:39:25.749394 23340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:25.733036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39344 (local address 127.7.132.129:46179)
0812 02:39:25.733706 (+ 670us) server_negotiation.cc:206] Beginning negotiation
0812 02:39:25.733713 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:39:25.733741 (+ 28us) server_negotiation.cc:407] Connection header received
0812 02:39:25.733802 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:39:25.733808 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:39:25.734018 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:39:25.734236 (+ 218us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:39:25.735928 (+ 1692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:25.737066 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:25.738470 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:25.739046 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:25.739145 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:39:25.743443 (+ 4298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:39:25.743455 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:39:25.743468 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:39:25.743506 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:39:25.746588 (+ 3082us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:39:25.747256 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:39:25.747263 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:39:25.747265 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:39:25.747448 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:39:25.747809 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:39:25.747817 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:39:25.747824 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:39:25.748303 (+ 479us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:39:25.748597 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0812 02:39:25.748784 (+ 187us) server_negotiation.cc:299] Negotiation successful
0812 02:39:25.748993 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":496,"thread_start_us":355,"threads_started":1}
I20260812 02:39:25.750811 23003 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:39:25.834424 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 22846
I20260812 02:39:25.855176 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 23018
Aug 12 02:39:25 dist-test-slave-mn4r krb5kdc[22169](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502361, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.132.130@KRBTEST.COM
I20260812 02:39:25.871544 23343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:25.856166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50224 (local address 127.7.132.130:37475)
0812 02:39:25.856662 (+ 496us) server_negotiation.cc:206] Beginning negotiation
0812 02:39:25.856670 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:39:25.856697 (+ 27us) server_negotiation.cc:407] Connection header received
0812 02:39:25.856769 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:39:25.856775 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:39:25.856998 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:39:25.857197 (+ 199us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:39:25.858481 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:25.859458 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:25.860673 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:25.861214 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:25.861317 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:39:25.865868 (+ 4551us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:39:25.865879 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:39:25.865894 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:39:25.865929 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:39:25.868888 (+ 2959us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:39:25.869541 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:39:25.869548 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:39:25.869551 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:39:25.869683 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:39:25.870021 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:39:25.870024 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:39:25.870026 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:39:25.870417 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:39:25.870670 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0812 02:39:25.870925 (+ 255us) server_negotiation.cc:299] Negotiation successful
0812 02:39:25.871140 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":247,"threads_started":1}
I20260812 02:39:25.872852 23176 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:39:25.955570 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 23018
I20260812 02:39:25.980562 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 23187
Aug 12 02:39:25 dist-test-slave-mn4r krb5kdc[22169](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502361, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.132.131@KRBTEST.COM
I20260812 02:39:25.997267 23345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:25.981514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60348 (local address 127.7.132.131:41889)
0812 02:39:25.982122 (+ 608us) server_negotiation.cc:206] Beginning negotiation
0812 02:39:25.982136 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 02:39:25.982166 (+ 30us) server_negotiation.cc:407] Connection header received
0812 02:39:25.982229 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:39:25.982235 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:39:25.982448 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:39:25.982578 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:39:25.983895 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:25.984972 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:25.986432 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:25.986998 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:25.987108 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:39:25.991752 (+ 4644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:39:25.991764 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:39:25.991779 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:39:25.991816 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:39:25.994803 (+ 2987us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:39:25.995476 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:39:25.995483 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:39:25.995485 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:39:25.995610 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:39:25.995957 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:39:25.995960 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:39:25.995962 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:39:25.996294 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:39:25.996537 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0812 02:39:25.996773 (+ 236us) server_negotiation.cc:299] Negotiation successful
0812 02:39:25.996968 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":413,"thread_start_us":245,"threads_started":1}
I20260812 02:39:25.998692 23330 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:39:26.083370 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 23187
I20260812 02:39:26.102926 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 22714
I20260812 02:39:26.104272 22787 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:39:26.214147 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 22714
I20260812 02:39:26.240412 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:40973
--webserver_interface=127.7.132.190
--webserver_port=32923
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:41657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.132.190:40973
--ranger_config_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.132.148:35801/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.7.132.148:35801/kms with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:39:26.586055 23350 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:39:26.586488 23350 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:39:26.586673 23350 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:39:26.593791 23350 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:39:26.593941 23350 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:39:26.594055 23350 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:39:26.594157 23350 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:39:26.594226 23350 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:39:26.604482 23350 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:41657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.7.132.148:35801/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.7.132.190:40973
--ranger_config_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:40973
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.132.190
--webserver_port=32923
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:39:26.605898 23350 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:39:26.607491 23350 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:39:26.619923 23357 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:39:26.619900 23358 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:39:26.620227 23361 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:39:26.620796 23350 server_base.cc:1034] running on GCE node
I20260812 02:39:26.621592 23350 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:39:26.623154 23350 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:39:26.624392 23350 hybrid_clock.cc:648] HybridClock initialized: now 1786502366624323 us; error 65 us; skew 500 ppm
I20260812 02:39:26.624768 23350 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:39:26 dist-test-slave-mn4r krb5kdc[22169](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502366, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:39:26.628131 23350 init.cc:377] Logged in from keytab as kudu/127.7.132.190@KRBTEST.COM (short username kudu)
I20260812 02:39:26.629626 23350 webserver.cc:460] Webserver started at http://127.7.132.190:32923/ using document root <none> and password file <none>
I20260812 02:39:26.630322 23350 fs_manager.cc:359] Metadata directory not provided
I20260812 02:39:26.630435 23350 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 02:39:26 dist-test-slave-mn4r krb5kdc[22169](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502366, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.190@KRBTEST.COM for HTTP/127.7.132.148@KRBTEST.COM
I20260812 02:39:26.683099 23350 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.004s sys 0.000s
I20260812 02:39:26.686524 23369 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:39:26.688247 23350 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260812 02:39:26.688460 23350 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/master-0/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/master-0/wal
uuid: "279e011032c94987b4f63d0ad2772a40"
format_stamp: "Formatted at 2026-08-12 02:39:22 on dist-test-slave-mn4r"
server_key: "a80a60b5a318f0cbb813b60b032762a5"
server_key_iv: "24584310b3a2685c1465d8f42ef3b3f4"
server_key_version: "kuduclusterkey@0"
I20260812 02:39:26.689491 23350 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:39:26.705734 23350 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:39:26.710048 23350 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:39:26.710382 23350 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:39:26.729379 23350 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.190:40973
I20260812 02:39:26.729403 23426 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.190:40973 every 8 connection(s)
I20260812 02:39:26.730994 23350 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
I20260812 02:39:26.738006 23427 sys_catalog.cc:263] Verifying existing consensus state
I20260812 02:39:26.739529 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 23350
I20260812 02:39:26.740388 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.132.148:35801/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.129:46179
--local_ip_for_outbound_sockets=127.7.132.129
--tserver_master_addrs=127.7.132.190:40973
--webserver_port=37567
--webserver_interface=127.7.132.129
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.129.keytab
--principal=kudu/127.7.132.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:41657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.7.132.148:35801/kms with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 02:39:26.741495 23427 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 279e011032c94987b4f63d0ad2772a40: Bootstrap starting.
I20260812 02:39:26.764006 23427 log.cc:824] T 00000000000000000000000000000000 P 279e011032c94987b4f63d0ad2772a40: Log is configured to *not* fsync() on all Append() calls
I20260812 02:39:26.777356 23427 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 279e011032c94987b4f63d0ad2772a40: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 02:39:26.778066 23427 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 279e011032c94987b4f63d0ad2772a40: Bootstrap complete.
I20260812 02:39:26.790402 23427 raft_consensus.cc:348] T 00000000000000000000000000000000 P 279e011032c94987b4f63d0ad2772a40 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "279e011032c94987b4f63d0ad2772a40" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 40973 } }
I20260812 02:39:26.791358 23427 raft_consensus.cc:729] T 00000000000000000000000000000000 P 279e011032c94987b4f63d0ad2772a40 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 279e011032c94987b4f63d0ad2772a40, State: Initialized, Role: FOLLOWER
I20260812 02:39:26.792058 23427 consensus_queue.cc:260] T 00000000000000000000000000000000 P 279e011032c94987b4f63d0ad2772a40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "279e011032c94987b4f63d0ad2772a40" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 40973 } }
I20260812 02:39:26.792308 23427 raft_consensus.cc:388] T 00000000000000000000000000000000 P 279e011032c94987b4f63d0ad2772a40 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 02:39:26.792441 23427 raft_consensus.cc:482] T 00000000000000000000000000000000 P 279e011032c94987b4f63d0ad2772a40 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 02:39:26.792631 23427 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 279e011032c94987b4f63d0ad2772a40 [term 1 FOLLOWER]: Advancing to term 2
I20260812 02:39:26.796744 23427 raft_consensus.cc:504] T 00000000000000000000000000000000 P 279e011032c94987b4f63d0ad2772a40 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "279e011032c94987b4f63d0ad2772a40" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 40973 } }
I20260812 02:39:26.797268 23427 leader_election.cc:302] T 00000000000000000000000000000000 P 279e011032c94987b4f63d0ad2772a40 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 279e011032c94987b4f63d0ad2772a40; no voters:
I20260812 02:39:26.797889 23427 leader_election.cc:288] T 00000000000000000000000000000000 P 279e011032c94987b4f63d0ad2772a40 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 02:39:26.798127 23432 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 279e011032c94987b4f63d0ad2772a40 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 02:39:26.799207 23432 raft_consensus.cc:686] T 00000000000000000000000000000000 P 279e011032c94987b4f63d0ad2772a40 [term 2 LEADER]: Becoming Leader. State: Replica: 279e011032c94987b4f63d0ad2772a40, State: Running, Role: LEADER
I20260812 02:39:26.800055 23427 sys_catalog.cc:564] T 00000000000000000000000000000000 P 279e011032c94987b4f63d0ad2772a40 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:39:26.799891 23432 consensus_queue.cc:237] T 00000000000000000000000000000000 P 279e011032c94987b4f63d0ad2772a40 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "279e011032c94987b4f63d0ad2772a40" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 40973 } }
I20260812 02:39:26.807058 23427 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260812 02:39:26.806839 23433 sys_catalog.cc:455] T 00000000000000000000000000000000 P 279e011032c94987b4f63d0ad2772a40 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "279e011032c94987b4f63d0ad2772a40" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "279e011032c94987b4f63d0ad2772a40" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 40973 } } }
I20260812 02:39:26.807564 23433 sys_catalog.cc:458] T 00000000000000000000000000000000 P 279e011032c94987b4f63d0ad2772a40 [sys.catalog]: This master's current role is: LEADER
I20260812 02:39:26.807327 23434 sys_catalog.cc:455] T 00000000000000000000000000000000 P 279e011032c94987b4f63d0ad2772a40 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 279e011032c94987b4f63d0ad2772a40. Latest consensus state: current_term: 2 leader_uuid: "279e011032c94987b4f63d0ad2772a40" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "279e011032c94987b4f63d0ad2772a40" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 40973 } } }
I20260812 02:39:26.807943 23434 sys_catalog.cc:458] T 00000000000000000000000000000000 P 279e011032c94987b4f63d0ad2772a40 [sys.catalog]: This master's current role is: LEADER
I20260812 02:39:26.809008 23450 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:39:26.827157 23450 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 02:39:26.836784 23450 catalog_manager.cc:1259] Loaded cluster ID: ebf0307fb5c946d1a54fc9bc835a2779
I20260812 02:39:26.837232 23450 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 02:39:26.842208 23450 catalog_manager.cc:1504] Loading token signing keys...
I20260812 02:39:26.844321 23450 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 279e011032c94987b4f63d0ad2772a40: Loaded TSK: 0
I20260812 02:39:26.846140 23450 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 02:39:27.228698 23429 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:39:27.229215 23429 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:39:27.229470 23429 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:39:27.241632 23429 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:39:27.241861 23429 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:39:27.242152 23429 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.129
I20260812 02:39:27.258803 23429 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:41657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.7.132.148:35801/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.129.keytab
--principal=kudu/127.7.132.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.129:46179
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.132.129
--webserver_port=37567
--tserver_master_addrs=127.7.132.190:40973
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.129
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:39:27.261066 23429 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:39:27.263233 23429 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:39:27.287142 23476 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:39:27.288553 23475 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:39:27.296325 23429 server_base.cc:1034] running on GCE node
W20260812 02:39:27.296468 23480 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:39:27.298995 23429 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:39:27.307790 23429 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:39:27.309209 23429 hybrid_clock.cc:648] HybridClock initialized: now 1786502367309135 us; error 79 us; skew 500 ppm
I20260812 02:39:27.309675 23429 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:39:27 dist-test-slave-mn4r krb5kdc[22169](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502367, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:39:27.314424 23429 init.cc:377] Logged in from keytab as kudu/127.7.132.129@KRBTEST.COM (short username kudu)
I20260812 02:39:27.319795 23429 webserver.cc:460] Webserver started at http://127.7.132.129:37567/ using document root <none> and password file <none>
I20260812 02:39:27.320924 23429 fs_manager.cc:359] Metadata directory not provided
I20260812 02:39:27.321174 23429 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 02:39:27 dist-test-slave-mn4r krb5kdc[22169](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502367, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.129@KRBTEST.COM for HTTP/127.7.132.148@KRBTEST.COM
I20260812 02:39:27.364578 23429 fs_manager.cc:709] Time spent opening directory manager: real 0.009s user 0.004s sys 0.004s
I20260812 02:39:27.372233 23492 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:39:27.374622 23429 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.006s sys 0.000s
I20260812 02:39:27.374936 23429 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/ts-0/wal
uuid: "7c97d93336a7454db0e8720dfd306be0"
format_stamp: "Formatted at 2026-08-12 02:39:23 on dist-test-slave-mn4r"
server_key: "fefdbdc6d1f90353d5feb85b8d470214"
server_key_iv: "9cc9e88ea5d1ac5bcf5dae3388a5c204"
server_key_version: "kuduclusterkey@0"
I20260812 02:39:27.376842 23429 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:39:27.430346 23429 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:39:27.435709 23429 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:39:27.436061 23429 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:39:27.437358 23429 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:39:27.439055 23429 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:39:27.439149 23429 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:39:27.439301 23429 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:39:27.439378 23429 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:39:27.496567 23429 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.129:46179
I20260812 02:39:27.496599 23621 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.129:46179 every 8 connection(s)
I20260812 02:39:27.498572 23429 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/ts-0/data/info.pb
I20260812 02:39:27.504500 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 23429
I20260812 02:39:27.505215 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.132.148:35801/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.130:37475
--local_ip_for_outbound_sockets=127.7.132.130
--tserver_master_addrs=127.7.132.190:40973
--webserver_port=45433
--webserver_interface=127.7.132.130
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.130.keytab
--principal=kudu/127.7.132.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:41657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.7.132.148:35801/kms with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 02:39:27 dist-test-slave-mn4r krb5kdc[22169](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502367, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.129@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 02:39:27.532922 23625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:27.501980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:40973 (local address 127.7.132.129:38801)
0812 02:39:27.503389 (+ 1409us) negotiation.cc:107] Waiting for socket to connect
0812 02:39:27.503454 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0812 02:39:27.506001 (+ 2547us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:39:27.508683 (+ 2682us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:39:27.508698 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:39:27.509401 (+ 703us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:39:27.510351 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:39:27.510382 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:39:27.513099 (+ 2717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:39:27.513107 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:39:27.515381 (+ 2274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:39:27.515400 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:39:27.517275 (+ 1875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:39:27.517285 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:39:27.517616 (+ 331us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:39:27.518565 (+ 949us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:39:27.518608 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:39:27.521617 (+ 3009us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:39:27.526922 (+ 5305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:39:27.526935 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:39:27.526955 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:39:27.527616 (+ 661us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:39:27.528241 (+ 625us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:39:27.528252 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:39:27.528258 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:39:27.528545 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:39:27.529657 (+ 1112us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:39:27.529668 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:39:27.530176 (+ 508us) client_negotiation.cc:769] Sending connection context
0812 02:39:27.530534 (+ 358us) client_negotiation.cc:240] Negotiation successful
0812 02:39:27.531283 (+ 749us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1085,"thread_start_us":788,"threads_started":1}
I20260812 02:39:27.536717 23624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:27.502030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:38801 (local address 127.7.132.190:40973)
0812 02:39:27.504270 (+ 2240us) server_negotiation.cc:206] Beginning negotiation
0812 02:39:27.504292 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0812 02:39:27.504353 (+ 61us) server_negotiation.cc:407] Connection header received
0812 02:39:27.506897 (+ 2544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:39:27.506922 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:39:27.507473 (+ 551us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:39:27.507936 (+ 463us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:39:27.510644 (+ 2708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:27.512866 (+ 2222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:27.515591 (+ 2725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:27.516764 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:27.516992 (+ 228us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:39:27.522001 (+ 5009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:39:27.522014 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:39:27.522039 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:39:27.522131 (+ 92us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:39:27.526721 (+ 4590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:39:27.527842 (+ 1121us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:39:27.527853 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:39:27.527864 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:39:27.528083 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:39:27.528731 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:39:27.528739 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:39:27.528745 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:39:27.529472 (+ 727us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:39:27.534351 (+ 4879us) server_negotiation.cc:1035] Waiting for connection context
0812 02:39:27.534683 (+ 332us) server_negotiation.cc:299] Negotiation successful
0812 02:39:27.535181 (+ 498us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1943,"thread_start_us":389,"threads_started":1}
I20260812 02:39:27.540992 23622 heartbeater.cc:347] Connected to a master server at 127.7.132.190:40973
I20260812 02:39:27.541352 23622 heartbeater.cc:464] Registering TS with master...
I20260812 02:39:27.542299 23622 heartbeater.cc:511] Master 127.7.132.190:40973 requested a full tablet report, sending...
W20260812 02:39:27.545696 23622 heartbeater.cc:644] Failed to heartbeat to 127.7.132.190:40973 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260812 02:39:27.545919 23622 heartbeater.cc:464] Registering TS with master...
I20260812 02:39:27.546216 23622 heartbeater.cc:511] Master 127.7.132.190:40973 requested a full tablet report, sending...
I20260812 02:39:27.547585 23622 heartbeater.cc:464] Registering TS with master...
I20260812 02:39:27.547899 23622 heartbeater.cc:511] Master 127.7.132.190:40973 requested a full tablet report, sending...
W20260812 02:39:27.549072 23622 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 02:39:27.955127 23627 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:39:27.955544 23627 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:39:27.955741 23627 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:39:27.967423 23627 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:39:27.967655 23627 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:39:27.967895 23627 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.130
I20260812 02:39:27.984162 23627 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:41657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.7.132.148:35801/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.130.keytab
--principal=kudu/127.7.132.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.130:37475
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.132.130
--webserver_port=45433
--tserver_master_addrs=127.7.132.190:40973
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.130
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:39:27.986129 23627 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:39:27.988055 23627 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:39:28.006546 23646 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:39:28.008396 23647 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:39:28.016100 23649 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:39:28.016691 23627 server_base.cc:1034] running on GCE node
I20260812 02:39:28.017524 23627 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:39:28.019757 23627 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:39:28.021085 23627 hybrid_clock.cc:648] HybridClock initialized: now 1786502368021003 us; error 67 us; skew 500 ppm
I20260812 02:39:28.021557 23627 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:39:28 dist-test-slave-mn4r krb5kdc[22169](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786502368, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 12 02:39:28 dist-test-slave-mn4r krb5kdc[22169](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502368, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:39:28.026327 23627 init.cc:377] Logged in from keytab as kudu/127.7.132.130@KRBTEST.COM (short username kudu)
I20260812 02:39:28.029699 23627 webserver.cc:460] Webserver started at http://127.7.132.130:45433/ using document root <none> and password file <none>
I20260812 02:39:28.030758 23627 fs_manager.cc:359] Metadata directory not provided
I20260812 02:39:28.030989 23627 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 02:39:28 dist-test-slave-mn4r krb5kdc[22169](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502368, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.130@KRBTEST.COM for HTTP/127.7.132.148@KRBTEST.COM
I20260812 02:39:28.073680 23627 fs_manager.cc:709] Time spent opening directory manager: real 0.020s user 0.011s sys 0.004s
I20260812 02:39:28.077986 23657 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:39:28.079931 23627 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260812 02:39:28.080124 23627 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/ts-1/wal
uuid: "20b9d93915064ca6b071511d633709cf"
format_stamp: "Formatted at 2026-08-12 02:39:24 on dist-test-slave-mn4r"
server_key: "1ede31db0ea993a311f74ef4a452f033"
server_key_iv: "c96d32683b301de116479f0f0937917e"
server_key_version: "kuduclusterkey@0"
I20260812 02:39:28.081585 23627 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:39:28.104746 23627 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:39:28.110050 23627 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:39:28.110435 23627 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:39:28.111620 23627 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:39:28.113219 23627 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:39:28.113343 23627 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:39:28.113485 23627 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:39:28.113588 23627 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:39:28.160202 23627 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.130:37475
I20260812 02:39:28.160257 23788 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.130:37475 every 8 connection(s)
I20260812 02:39:28.161801 23627 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/ts-1/data/info.pb
I20260812 02:39:28.164121 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 23627
I20260812 02:39:28.164992 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.132.148:35801/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.131:41889
--local_ip_for_outbound_sockets=127.7.132.131
--tserver_master_addrs=127.7.132.190:40973
--webserver_port=40705
--webserver_interface=127.7.132.131
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.131.keytab
--principal=kudu/127.7.132.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:41657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.7.132.148:35801/kms with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 02:39:28 dist-test-slave-mn4r krb5kdc[22169](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502368, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.130@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 02:39:28.224454 23791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:28.166662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:34555 (local address 127.7.132.190:40973)
0812 02:39:28.188541 (+ 21879us) server_negotiation.cc:206] Beginning negotiation
0812 02:39:28.188552 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 02:39:28.191246 (+ 2694us) server_negotiation.cc:407] Connection header received
0812 02:39:28.195326 (+ 4080us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:39:28.195336 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:39:28.195475 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:39:28.195582 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:39:28.198790 (+ 3208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:28.200383 (+ 1593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:28.209080 (+ 8697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:28.209781 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:28.209906 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:39:28.215170 (+ 5264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:39:28.215179 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:39:28.215187 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:39:28.215237 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:39:28.219506 (+ 4269us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:39:28.220723 (+ 1217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:39:28.220733 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:39:28.220739 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:39:28.220914 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:39:28.221583 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:39:28.221591 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:39:28.221597 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:39:28.222185 (+ 588us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:39:28.222355 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0812 02:39:28.223587 (+ 1232us) server_negotiation.cc:299] Negotiation successful
0812 02:39:28.223887 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21658,"thread_start_us":24588,"threads_started":1}
I20260812 02:39:28.227751 23792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:28.167348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:40973 (local address 127.7.132.130:34555)
0812 02:39:28.189402 (+ 22054us) negotiation.cc:107] Waiting for socket to connect
0812 02:39:28.189481 (+ 79us) client_negotiation.cc:174] Beginning negotiation
0812 02:39:28.194854 (+ 5373us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:39:28.196374 (+ 1520us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:39:28.196391 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:39:28.197280 (+ 889us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:39:28.198516 (+ 1236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:39:28.198549 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:39:28.200666 (+ 2117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:39:28.200674 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:39:28.208827 (+ 8153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:39:28.208850 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:39:28.210379 (+ 1529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:39:28.210390 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:39:28.210748 (+ 358us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:39:28.211675 (+ 927us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:39:28.211721 (+ 46us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:39:28.214828 (+ 3107us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:39:28.219730 (+ 4902us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:39:28.219742 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:39:28.219767 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:39:28.220543 (+ 776us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:39:28.221120 (+ 577us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:39:28.221127 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:39:28.221133 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:39:28.221424 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:39:28.222473 (+ 1049us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:39:28.222483 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:39:28.223028 (+ 545us) client_negotiation.cc:769] Sending connection context
0812 02:39:28.225151 (+ 2123us) client_negotiation.cc:240] Negotiation successful
0812 02:39:28.225988 (+ 837us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21577,"thread_start_us":22589,"threads_started":1}
I20260812 02:39:28.229897 23789 heartbeater.cc:347] Connected to a master server at 127.7.132.190:40973
I20260812 02:39:28.230264 23789 heartbeater.cc:464] Registering TS with master...
I20260812 02:39:28.231181 23789 heartbeater.cc:511] Master 127.7.132.190:40973 requested a full tablet report, sending...
I20260812 02:39:28.233402 23387 ts_manager.cc:194] Registered new tserver with Master: 20b9d93915064ca6b071511d633709cf (127.7.132.130:37475)
I20260812 02:39:28.235808 23387 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.130@KRBTEST.COM'} at 127.7.132.130:34555
Aug 12 02:39:28 dist-test-slave-mn4r krb5kdc[22169](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502368, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.190@KRBTEST.COM for HTTP/127.7.132.148@KRBTEST.COM
I20260812 02:39:28.549261 23622 heartbeater.cc:464] Registering TS with master...
I20260812 02:39:28.549695 23622 heartbeater.cc:511] Master 127.7.132.190:40973 requested a full tablet report, sending...
I20260812 02:39:28.551072 23387 ts_manager.cc:194] Registered new tserver with Master: 7c97d93336a7454db0e8720dfd306be0 (127.7.132.129:46179)
I20260812 02:39:28.552186 23387 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.129@KRBTEST.COM'} at 127.7.132.129:38801
W20260812 02:39:28.676646 23793 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:39:28.677177 23793 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:39:28.677376 23793 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:39:28.686293 23793 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:39:28.686518 23793 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:39:28.686712 23793 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.131
I20260812 02:39:28.701982 23793 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:41657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.7.132.148:35801/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.131.keytab
--principal=kudu/127.7.132.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.131:41889
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.132.131
--webserver_port=40705
--tserver_master_addrs=127.7.132.190:40973
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.131
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:39:28.704123 23793 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:39:28.706007 23793 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:39:28.745867 23800 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:39:28.745898 23799 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:39:28.747905 23793 server_base.cc:1034] running on GCE node
W20260812 02:39:28.748407 23802 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:39:28.749238 23793 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:39:28.750551 23793 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:39:28.754225 23793 hybrid_clock.cc:648] HybridClock initialized: now 1786502368754121 us; error 69 us; skew 500 ppm
I20260812 02:39:28.754712 23793 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:39:28 dist-test-slave-mn4r krb5kdc[22169](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502368, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:39:28.759642 23793 init.cc:377] Logged in from keytab as kudu/127.7.132.131@KRBTEST.COM (short username kudu)
I20260812 02:39:28.786603 23793 webserver.cc:460] Webserver started at http://127.7.132.131:40705/ using document root <none> and password file <none>
I20260812 02:39:28.787863 23793 fs_manager.cc:359] Metadata directory not provided
I20260812 02:39:28.788094 23793 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 02:39:28 dist-test-slave-mn4r krb5kdc[22169](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502368, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.131@KRBTEST.COM for HTTP/127.7.132.148@KRBTEST.COM
I20260812 02:39:28.845822 23793 fs_manager.cc:709] Time spent opening directory manager: real 0.015s user 0.005s sys 0.008s
I20260812 02:39:28.849937 23812 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:39:28.852598 23793 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.000s
I20260812 02:39:28.852807 23793 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/ts-2/wal
uuid: "4126d927aac74375b2d65068c2420061"
format_stamp: "Formatted at 2026-08-12 02:39:25 on dist-test-slave-mn4r"
server_key: "5efe139cc757fbaf8439fe8bf5c6f75f"
server_key_iv: "b22dbd3ade37610e147dfbbb783afb04"
server_key_version: "kuduclusterkey@0"
I20260812 02:39:28.854244 23793 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
Aug 12 02:39:28 dist-test-slave-mn4r krb5kdc[22169](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502368, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.190@KRBTEST.COM for HTTP/127.7.132.148@KRBTEST.COM
I20260812 02:39:28.904369 23793 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:39:28.909816 23793 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:39:28.910254 23793 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:39:28.917722 23793 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:39:28.927455 23793 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:39:28.927657 23793 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:39:28.927809 23793 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:39:28.927912 23793 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:39:29.240760 23789 heartbeater.cc:503] Master 127.7.132.190:40973 was elected leader, sending a full tablet report...
I20260812 02:39:29.334224 23793 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.131:41889
I20260812 02:39:29.334295 23956 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.131:41889 every 8 connection(s)
I20260812 02:39:29.335937 23793 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/ts-2/data/info.pb
I20260812 02:39:29.337158 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 23793
I20260812 02:39:29.384111 23960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:29.338252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34452 (local address 127.7.132.190:40973)
0812 02:39:29.356571 (+ 18319us) server_negotiation.cc:206] Beginning negotiation
0812 02:39:29.356578 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:39:29.357515 (+ 937us) server_negotiation.cc:407] Connection header received
0812 02:39:29.357596 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:39:29.357601 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:39:29.357697 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:39:29.357800 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:39:29.361496 (+ 3696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:29.362844 (+ 1348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:29.365361 (+ 2517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:29.366119 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:29.366260 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:39:29.376545 (+ 10285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:39:29.376551 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:39:29.376555 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:39:29.376610 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:39:29.380532 (+ 3922us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:39:29.381478 (+ 946us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:39:29.381484 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:39:29.381488 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:39:29.381645 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:39:29.382177 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:39:29.382182 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:39:29.382185 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:39:29.382565 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:39:29.382833 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0812 02:39:29.383486 (+ 653us) server_negotiation.cc:299] Negotiation successful
0812 02:39:29.383725 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18161,"thread_start_us":19133,"threads_started":1}
Aug 12 02:39:29 dist-test-slave-mn4r krb5kdc[22169](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502368, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.131@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 02:39:29.394160 23964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:29.340899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:56807 (local address 127.7.132.190:40973)
0812 02:39:29.371962 (+ 31063us) server_negotiation.cc:206] Beginning negotiation
0812 02:39:29.371972 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 02:39:29.375888 (+ 3916us) server_negotiation.cc:407] Connection header received
0812 02:39:29.375971 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:39:29.375977 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:39:29.376092 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:39:29.376205 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:39:29.378729 (+ 2524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:29.380219 (+ 1490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:29.382312 (+ 2093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:29.383007 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:29.383147 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:39:29.387868 (+ 4721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:39:29.387875 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:39:29.387880 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:39:29.387925 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:39:29.390838 (+ 2913us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:39:29.391594 (+ 756us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:39:29.391599 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:39:29.391603 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:39:29.391730 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:39:29.392178 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:39:29.392183 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:39:29.392186 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:39:29.392451 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:39:29.392610 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0812 02:39:29.393413 (+ 803us) server_negotiation.cc:299] Negotiation successful
0812 02:39:29.393648 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30867,"thread_start_us":30690,"threads_started":1}
I20260812 02:39:29.395221 23963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:29.340153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:40973 (local address 127.7.132.131:56807)
0812 02:39:29.372648 (+ 32495us) negotiation.cc:107] Waiting for socket to connect
0812 02:39:29.372721 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0812 02:39:29.375396 (+ 2675us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:39:29.376805 (+ 1409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:39:29.376821 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:39:29.377519 (+ 698us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:39:29.378516 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:39:29.378548 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:39:29.380445 (+ 1897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:39:29.380452 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:39:29.382142 (+ 1690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:39:29.382161 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:39:29.383262 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:39:29.383270 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:39:29.383604 (+ 334us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:39:29.384582 (+ 978us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:39:29.384607 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:39:29.387554 (+ 2947us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:39:29.390990 (+ 3436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:39:29.391000 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:39:29.391017 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:39:29.391438 (+ 421us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:39:29.391846 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:39:29.391851 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:39:29.391855 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:39:29.392053 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:39:29.392590 (+ 537us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:39:29.392599 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:39:29.393028 (+ 429us) client_negotiation.cc:769] Sending connection context
0812 02:39:29.393305 (+ 277us) client_negotiation.cc:240] Negotiation successful
0812 02:39:29.393870 (+ 565us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32156,"thread_start_us":31881,"threads_started":1}
I20260812 02:39:29.396792 23958 heartbeater.cc:347] Connected to a master server at 127.7.132.190:40973
I20260812 02:39:29.397123 23958 heartbeater.cc:464] Registering TS with master...
I20260812 02:39:29.397867 23958 heartbeater.cc:511] Master 127.7.132.190:40973 requested a full tablet report, sending...
I20260812 02:39:29.399737 23385 ts_manager.cc:194] Registered new tserver with Master: 4126d927aac74375b2d65068c2420061 (127.7.132.131:41889)
I20260812 02:39:29.400823 7698 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 02:39:29.401104 23385 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.131@KRBTEST.COM'} at 127.7.132.131:56807
I20260812 02:39:29.419144 23964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:29.405507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34460 (local address 127.7.132.190:40973)
0812 02:39:29.405690 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0812 02:39:29.405694 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 02:39:29.406159 (+ 465us) server_negotiation.cc:407] Connection header received
0812 02:39:29.406371 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:39:29.406376 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:39:29.406451 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:39:29.406557 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:39:29.408155 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:29.408923 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:29.410385 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:29.410817 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:29.410955 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:39:29.413664 (+ 2709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:39:29.413675 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:39:29.413678 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:39:29.413709 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:39:29.416425 (+ 2716us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:39:29.417129 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:39:29.417135 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:39:29.417138 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:39:29.417270 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:39:29.417661 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:39:29.417666 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:39:29.417670 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:39:29.417940 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:39:29.418221 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0812 02:39:29.418547 (+ 326us) server_negotiation.cc:299] Negotiation successful
0812 02:39:29.418780 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260812 02:39:29.424638 23385 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34460:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 02:39:29.432440 23385 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 02:39:29.486483 23976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:29.448020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:46179 (local address 127.0.0.1:36160)
0812 02:39:29.476414 (+ 28394us) negotiation.cc:107] Waiting for socket to connect
0812 02:39:29.476440 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0812 02:39:29.476582 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:39:29.478372 (+ 1790us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:39:29.478377 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:39:29.478401 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:39:29.478771 (+ 370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:39:29.478782 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:39:29.480964 (+ 2182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:39:29.480969 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:39:29.483502 (+ 2533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:39:29.483518 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:39:29.485441 (+ 1923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:39:29.485448 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:39:29.485651 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:39:29.485705 (+ 54us) client_negotiation.cc:769] Sending connection context
0812 02:39:29.485809 (+ 104us) client_negotiation.cc:240] Negotiation successful
0812 02:39:29.486004 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28232,"mutex_wait_us":1,"thread_start_us":29043,"threads_started":1}
I20260812 02:39:29.487000 23980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:29.447966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.130:37475 (local address 127.0.0.1:55350)
0812 02:39:29.471527 (+ 23561us) negotiation.cc:107] Waiting for socket to connect
0812 02:39:29.471588 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0812 02:39:29.471764 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:39:29.474145 (+ 2381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:39:29.474153 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:39:29.474187 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:39:29.474612 (+ 425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:39:29.474625 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:39:29.481798 (+ 7173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:39:29.481807 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:39:29.484197 (+ 2390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:39:29.484211 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:39:29.486214 (+ 2003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:39:29.486222 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:39:29.486390 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:39:29.486438 (+ 48us) client_negotiation.cc:769] Sending connection context
0812 02:39:29.486509 (+ 71us) client_negotiation.cc:240] Negotiation successful
0812 02:39:29.486640 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21860,"thread_start_us":28714,"threads_started":1}
I20260812 02:39:29.488432 23979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:29.448637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36160 (local address 127.7.132.129:46179)
0812 02:39:29.477753 (+ 29116us) server_negotiation.cc:206] Beginning negotiation
0812 02:39:29.477761 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:39:29.477790 (+ 29us) server_negotiation.cc:407] Connection header received
0812 02:39:29.477880 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:39:29.477887 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:39:29.478156 (+ 269us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:39:29.478967 (+ 811us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:39:29.479212 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:29.480781 (+ 1569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:29.483723 (+ 2942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:29.485191 (+ 1468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:29.487435 (+ 2244us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:39:29.487736 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0812 02:39:29.487804 (+ 68us) server_negotiation.cc:299] Negotiation successful
0812 02:39:29.488040 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28937,"thread_start_us":29058,"threads_started":1}
I20260812 02:39:29.488498 23978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:29.448573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55350 (local address 127.7.132.130:37475)
0812 02:39:29.473279 (+ 24706us) server_negotiation.cc:206] Beginning negotiation
0812 02:39:29.473293 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 02:39:29.473334 (+ 41us) server_negotiation.cc:407] Connection header received
0812 02:39:29.473466 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:39:29.473479 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:39:29.473831 (+ 352us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:39:29.474134 (+ 303us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:39:29.479837 (+ 5703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:29.481473 (+ 1636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:29.484422 (+ 2949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:29.485936 (+ 1514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:29.487603 (+ 1667us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:39:29.487847 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0812 02:39:29.487921 (+ 74us) server_negotiation.cc:299] Negotiation successful
0812 02:39:29.488098 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24492,"thread_start_us":24295,"threads_started":1}
I20260812 02:39:29.489620 23977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:29.450504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.131:41889 (local address 127.0.0.1:56348)
0812 02:39:29.477392 (+ 26888us) negotiation.cc:107] Waiting for socket to connect
0812 02:39:29.477413 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 02:39:29.477544 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:39:29.480708 (+ 3164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:39:29.480713 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:39:29.480737 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:39:29.481145 (+ 408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:39:29.481156 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:39:29.483376 (+ 2220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:39:29.483381 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:39:29.485545 (+ 2164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:39:29.485561 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:39:29.488874 (+ 3313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:39:29.488880 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:39:29.489066 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:39:29.489096 (+ 30us) client_negotiation.cc:769] Sending connection context
0812 02:39:29.489162 (+ 66us) client_negotiation.cc:240] Negotiation successful
0812 02:39:29.489296 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26734,"thread_start_us":19163,"threads_started":1}
I20260812 02:39:29.490859 23981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:29.451239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56348 (local address 127.7.132.131:41889)
0812 02:39:29.479804 (+ 28565us) server_negotiation.cc:206] Beginning negotiation
0812 02:39:29.479814 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 02:39:29.479841 (+ 27us) server_negotiation.cc:407] Connection header received
0812 02:39:29.479955 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:39:29.479963 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:39:29.480207 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:39:29.480399 (+ 192us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:39:29.481369 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:29.483225 (+ 1856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:29.485780 (+ 2555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:29.488645 (+ 2865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:29.489991 (+ 1346us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:39:29.490244 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0812 02:39:29.490316 (+ 72us) server_negotiation.cc:299] Negotiation successful
0812 02:39:29.490494 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28398,"thread_start_us":28239,"threads_started":1}
I20260812 02:39:29.492569 23545 tablet_service.cc:1467] Processing CreateTablet for tablet 0a6d4acbe87745608b0b12577d445f5b (DEFAULT_TABLE table=test-table [id=de9feaac0edc4f8985afd848155ab93a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:39:29.492587 23715 tablet_service.cc:1467] Processing CreateTablet for tablet 0a6d4acbe87745608b0b12577d445f5b (DEFAULT_TABLE table=test-table [id=de9feaac0edc4f8985afd848155ab93a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:39:29.494154 23715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a6d4acbe87745608b0b12577d445f5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:39:29.494609 23545 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a6d4acbe87745608b0b12577d445f5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:39:29.494858 23871 tablet_service.cc:1467] Processing CreateTablet for tablet 0a6d4acbe87745608b0b12577d445f5b (DEFAULT_TABLE table=test-table [id=de9feaac0edc4f8985afd848155ab93a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:39:29.496359 23871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a6d4acbe87745608b0b12577d445f5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:39:29.517813 23982 tablet_bootstrap.cc:492] T 0a6d4acbe87745608b0b12577d445f5b P 7c97d93336a7454db0e8720dfd306be0: Bootstrap starting.
I20260812 02:39:29.523871 23982 tablet_bootstrap.cc:654] T 0a6d4acbe87745608b0b12577d445f5b P 7c97d93336a7454db0e8720dfd306be0: Neither blocks nor log segments found. Creating new log.
I20260812 02:39:29.525329 23982 log.cc:824] T 0a6d4acbe87745608b0b12577d445f5b P 7c97d93336a7454db0e8720dfd306be0: Log is configured to *not* fsync() on all Append() calls
I20260812 02:39:29.527016 23984 tablet_bootstrap.cc:492] T 0a6d4acbe87745608b0b12577d445f5b P 4126d927aac74375b2d65068c2420061: Bootstrap starting.
I20260812 02:39:29.531996 23984 tablet_bootstrap.cc:654] T 0a6d4acbe87745608b0b12577d445f5b P 4126d927aac74375b2d65068c2420061: Neither blocks nor log segments found. Creating new log.
I20260812 02:39:29.533468 23984 log.cc:824] T 0a6d4acbe87745608b0b12577d445f5b P 4126d927aac74375b2d65068c2420061: Log is configured to *not* fsync() on all Append() calls
I20260812 02:39:29.535215 23982 tablet_bootstrap.cc:492] T 0a6d4acbe87745608b0b12577d445f5b P 7c97d93336a7454db0e8720dfd306be0: No bootstrap required, opened a new log
I20260812 02:39:29.535636 23982 ts_tablet_manager.cc:1397] T 0a6d4acbe87745608b0b12577d445f5b P 7c97d93336a7454db0e8720dfd306be0: Time spent bootstrapping tablet: real 0.018s user 0.010s sys 0.004s
I20260812 02:39:29.542886 23986 tablet_bootstrap.cc:492] T 0a6d4acbe87745608b0b12577d445f5b P 20b9d93915064ca6b071511d633709cf: Bootstrap starting.
I20260812 02:39:29.542950 23984 tablet_bootstrap.cc:492] T 0a6d4acbe87745608b0b12577d445f5b P 4126d927aac74375b2d65068c2420061: No bootstrap required, opened a new log
I20260812 02:39:29.543217 23984 ts_tablet_manager.cc:1397] T 0a6d4acbe87745608b0b12577d445f5b P 4126d927aac74375b2d65068c2420061: Time spent bootstrapping tablet: real 0.017s user 0.003s sys 0.011s
I20260812 02:39:29.546649 23986 tablet_bootstrap.cc:654] T 0a6d4acbe87745608b0b12577d445f5b P 20b9d93915064ca6b071511d633709cf: Neither blocks nor log segments found. Creating new log.
I20260812 02:39:29.547987 23982 raft_consensus.cc:348] T 0a6d4acbe87745608b0b12577d445f5b P 7c97d93336a7454db0e8720dfd306be0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c97d93336a7454db0e8720dfd306be0" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 46179 } } peers { permanent_uuid: "20b9d93915064ca6b071511d633709cf" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37475 } } peers { permanent_uuid: "4126d927aac74375b2d65068c2420061" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 41889 } }
I20260812 02:39:29.548516 23982 raft_consensus.cc:374] T 0a6d4acbe87745608b0b12577d445f5b P 7c97d93336a7454db0e8720dfd306be0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:39:29.548661 23982 raft_consensus.cc:729] T 0a6d4acbe87745608b0b12577d445f5b P 7c97d93336a7454db0e8720dfd306be0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c97d93336a7454db0e8720dfd306be0, State: Initialized, Role: FOLLOWER
I20260812 02:39:29.549353 23982 consensus_queue.cc:260] T 0a6d4acbe87745608b0b12577d445f5b P 7c97d93336a7454db0e8720dfd306be0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c97d93336a7454db0e8720dfd306be0" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 46179 } } peers { permanent_uuid: "20b9d93915064ca6b071511d633709cf" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37475 } } peers { permanent_uuid: "4126d927aac74375b2d65068c2420061" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 41889 } }
I20260812 02:39:29.549392 23986 log.cc:824] T 0a6d4acbe87745608b0b12577d445f5b P 20b9d93915064ca6b071511d633709cf: Log is configured to *not* fsync() on all Append() calls
I20260812 02:39:29.555555 23622 heartbeater.cc:503] Master 127.7.132.190:40973 was elected leader, sending a full tablet report...
I20260812 02:39:29.562353 23984 raft_consensus.cc:348] T 0a6d4acbe87745608b0b12577d445f5b P 4126d927aac74375b2d65068c2420061 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c97d93336a7454db0e8720dfd306be0" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 46179 } } peers { permanent_uuid: "20b9d93915064ca6b071511d633709cf" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37475 } } peers { permanent_uuid: "4126d927aac74375b2d65068c2420061" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 41889 } }
I20260812 02:39:29.563060 23984 raft_consensus.cc:374] T 0a6d4acbe87745608b0b12577d445f5b P 4126d927aac74375b2d65068c2420061 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:39:29.563236 23984 raft_consensus.cc:729] T 0a6d4acbe87745608b0b12577d445f5b P 4126d927aac74375b2d65068c2420061 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4126d927aac74375b2d65068c2420061, State: Initialized, Role: FOLLOWER
I20260812 02:39:29.564221 23984 consensus_queue.cc:260] T 0a6d4acbe87745608b0b12577d445f5b P 4126d927aac74375b2d65068c2420061 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c97d93336a7454db0e8720dfd306be0" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 46179 } } peers { permanent_uuid: "20b9d93915064ca6b071511d633709cf" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37475 } } peers { permanent_uuid: "4126d927aac74375b2d65068c2420061" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 41889 } }
I20260812 02:39:29.566473 23982 ts_tablet_manager.cc:1428] T 0a6d4acbe87745608b0b12577d445f5b P 7c97d93336a7454db0e8720dfd306be0: Time spent starting tablet: real 0.031s user 0.018s sys 0.012s
I20260812 02:39:29.567379 23958 heartbeater.cc:503] Master 127.7.132.190:40973 was elected leader, sending a full tablet report...
I20260812 02:39:29.568039 23984 ts_tablet_manager.cc:1428] T 0a6d4acbe87745608b0b12577d445f5b P 4126d927aac74375b2d65068c2420061: Time spent starting tablet: real 0.025s user 0.013s sys 0.003s
I20260812 02:39:29.568470 23986 tablet_bootstrap.cc:492] T 0a6d4acbe87745608b0b12577d445f5b P 20b9d93915064ca6b071511d633709cf: No bootstrap required, opened a new log
I20260812 02:39:29.568768 23986 ts_tablet_manager.cc:1397] T 0a6d4acbe87745608b0b12577d445f5b P 20b9d93915064ca6b071511d633709cf: Time spent bootstrapping tablet: real 0.026s user 0.014s sys 0.010s
I20260812 02:39:29.578362 23986 raft_consensus.cc:348] T 0a6d4acbe87745608b0b12577d445f5b P 20b9d93915064ca6b071511d633709cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c97d93336a7454db0e8720dfd306be0" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 46179 } } peers { permanent_uuid: "20b9d93915064ca6b071511d633709cf" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37475 } } peers { permanent_uuid: "4126d927aac74375b2d65068c2420061" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 41889 } }
I20260812 02:39:29.578737 23986 raft_consensus.cc:374] T 0a6d4acbe87745608b0b12577d445f5b P 20b9d93915064ca6b071511d633709cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:39:29.578835 23986 raft_consensus.cc:729] T 0a6d4acbe87745608b0b12577d445f5b P 20b9d93915064ca6b071511d633709cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20b9d93915064ca6b071511d633709cf, State: Initialized, Role: FOLLOWER
I20260812 02:39:29.579480 23986 consensus_queue.cc:260] T 0a6d4acbe87745608b0b12577d445f5b P 20b9d93915064ca6b071511d633709cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c97d93336a7454db0e8720dfd306be0" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 46179 } } peers { permanent_uuid: "20b9d93915064ca6b071511d633709cf" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37475 } } peers { permanent_uuid: "4126d927aac74375b2d65068c2420061" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 41889 } }
I20260812 02:39:29.580830 23986 ts_tablet_manager.cc:1428] T 0a6d4acbe87745608b0b12577d445f5b P 20b9d93915064ca6b071511d633709cf: Time spent starting tablet: real 0.012s user 0.010s sys 0.000s
W20260812 02:39:29.610615 23959 tablet.cc:2369] T 0a6d4acbe87745608b0b12577d445f5b P 4126d927aac74375b2d65068c2420061: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 02:39:29.645376 23991 raft_consensus.cc:482] T 0a6d4acbe87745608b0b12577d445f5b P 4126d927aac74375b2d65068c2420061 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 02:39:29.645668 23991 raft_consensus.cc:504] T 0a6d4acbe87745608b0b12577d445f5b P 4126d927aac74375b2d65068c2420061 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c97d93336a7454db0e8720dfd306be0" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 46179 } } peers { permanent_uuid: "20b9d93915064ca6b071511d633709cf" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37475 } } peers { permanent_uuid: "4126d927aac74375b2d65068c2420061" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 41889 } }
I20260812 02:39:29.647433 23991 leader_election.cc:288] T 0a6d4acbe87745608b0b12577d445f5b P 4126d927aac74375b2d65068c2420061 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c97d93336a7454db0e8720dfd306be0 (127.7.132.129:46179), 20b9d93915064ca6b071511d633709cf (127.7.132.130:37475)
I20260812 02:39:29.654279 23979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:29.647923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:41255 (local address 127.7.132.129:46179)
0812 02:39:29.648137 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0812 02:39:29.648144 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:39:29.648165 (+ 21us) server_negotiation.cc:407] Connection header received
0812 02:39:29.648256 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:39:29.648261 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:39:29.648354 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:39:29.648457 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:39:29.649000 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:29.650044 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:29.652084 (+ 2040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:29.653283 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:29.653407 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:39:29.653462 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 02:39:29.653918 (+ 456us) server_negotiation.cc:299] Negotiation successful
0812 02:39:29.654014 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260812 02:39:29.654412 23963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:29.647744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:46179 (local address 127.7.132.131:41255)
0812 02:39:29.647955 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0812 02:39:29.647977 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 02:39:29.648136 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:39:29.648586 (+ 450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:39:29.648596 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:39:29.648632 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:39:29.648855 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:39:29.648864 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:39:29.650215 (+ 1351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:39:29.650221 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:39:29.651901 (+ 1680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:39:29.651915 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:39:29.653482 (+ 1567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:39:29.653490 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:39:29.653713 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:39:29.653755 (+ 42us) client_negotiation.cc:769] Sending connection context
0812 02:39:29.653894 (+ 139us) client_negotiation.cc:240] Negotiation successful
0812 02:39:29.654053 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":75,"mutex_wait_us":24}
I20260812 02:39:29.655490 23993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:29.647826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.130:37475 (local address 127.7.132.131:50781)
0812 02:39:29.648739 (+ 913us) negotiation.cc:107] Waiting for socket to connect
0812 02:39:29.648753 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 02:39:29.648912 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:39:29.649284 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:39:29.649289 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:39:29.649316 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:39:29.649684 (+ 368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:39:29.649695 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:39:29.651117 (+ 1422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:39:29.651121 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:39:29.652914 (+ 1793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:39:29.652929 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:39:29.654791 (+ 1862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:39:29.654800 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:39:29.654973 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:39:29.655013 (+ 40us) client_negotiation.cc:769] Sending connection context
0812 02:39:29.655123 (+ 110us) client_negotiation.cc:240] Negotiation successful
0812 02:39:29.655215 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":790,"mutex_wait_us":42,"thread_start_us":565,"threads_started":1}
I20260812 02:39:29.655821 23978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:29.648215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:50781 (local address 127.7.132.130:37475)
0812 02:39:29.648467 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0812 02:39:29.648472 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:39:29.648862 (+ 390us) server_negotiation.cc:407] Connection header received
0812 02:39:29.649083 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:39:29.649087 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:39:29.649159 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:39:29.649232 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:39:29.649840 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:29.650935 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:29.653105 (+ 2170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:29.654556 (+ 1451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:29.654751 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:39:29.654828 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 02:39:29.655182 (+ 354us) server_negotiation.cc:299] Negotiation successful
0812 02:39:29.655345 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260812 02:39:29.655723 23567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a6d4acbe87745608b0b12577d445f5b" candidate_uuid: "4126d927aac74375b2d65068c2420061" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c97d93336a7454db0e8720dfd306be0" is_pre_election: true
I20260812 02:39:29.656426 23567 raft_consensus.cc:2454] T 0a6d4acbe87745608b0b12577d445f5b P 7c97d93336a7454db0e8720dfd306be0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4126d927aac74375b2d65068c2420061 in term 0.
I20260812 02:39:29.656857 23739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a6d4acbe87745608b0b12577d445f5b" candidate_uuid: "4126d927aac74375b2d65068c2420061" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20b9d93915064ca6b071511d633709cf" is_pre_election: true
I20260812 02:39:29.657297 23817 leader_election.cc:302] T 0a6d4acbe87745608b0b12577d445f5b P 4126d927aac74375b2d65068c2420061 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4126d927aac74375b2d65068c2420061, 7c97d93336a7454db0e8720dfd306be0; no voters:
I20260812 02:39:29.657433 23739 raft_consensus.cc:2454] T 0a6d4acbe87745608b0b12577d445f5b P 20b9d93915064ca6b071511d633709cf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4126d927aac74375b2d65068c2420061 in term 0.
I20260812 02:39:29.657742 23991 raft_consensus.cc:2781] T 0a6d4acbe87745608b0b12577d445f5b P 4126d927aac74375b2d65068c2420061 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 02:39:29.657967 23991 raft_consensus.cc:482] T 0a6d4acbe87745608b0b12577d445f5b P 4126d927aac74375b2d65068c2420061 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 02:39:29.658108 23991 raft_consensus.cc:3037] T 0a6d4acbe87745608b0b12577d445f5b P 4126d927aac74375b2d65068c2420061 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:39:29.660439 23991 raft_consensus.cc:504] T 0a6d4acbe87745608b0b12577d445f5b P 4126d927aac74375b2d65068c2420061 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c97d93336a7454db0e8720dfd306be0" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 46179 } } peers { permanent_uuid: "20b9d93915064ca6b071511d633709cf" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37475 } } peers { permanent_uuid: "4126d927aac74375b2d65068c2420061" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 41889 } }
I20260812 02:39:29.661095 23991 leader_election.cc:288] T 0a6d4acbe87745608b0b12577d445f5b P 4126d927aac74375b2d65068c2420061 [CANDIDATE]: Term 1 election: Requested vote from peers 7c97d93336a7454db0e8720dfd306be0 (127.7.132.129:46179), 20b9d93915064ca6b071511d633709cf (127.7.132.130:37475)
I20260812 02:39:29.661682 23739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a6d4acbe87745608b0b12577d445f5b" candidate_uuid: "4126d927aac74375b2d65068c2420061" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20b9d93915064ca6b071511d633709cf"
I20260812 02:39:29.661682 23567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a6d4acbe87745608b0b12577d445f5b" candidate_uuid: "4126d927aac74375b2d65068c2420061" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c97d93336a7454db0e8720dfd306be0"
I20260812 02:39:29.661918 23739 raft_consensus.cc:3037] T 0a6d4acbe87745608b0b12577d445f5b P 20b9d93915064ca6b071511d633709cf [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:39:29.661919 23567 raft_consensus.cc:3037] T 0a6d4acbe87745608b0b12577d445f5b P 7c97d93336a7454db0e8720dfd306be0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:39:29.665401 23567 raft_consensus.cc:2454] T 0a6d4acbe87745608b0b12577d445f5b P 7c97d93336a7454db0e8720dfd306be0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4126d927aac74375b2d65068c2420061 in term 1.
I20260812 02:39:29.665403 23739 raft_consensus.cc:2454] T 0a6d4acbe87745608b0b12577d445f5b P 20b9d93915064ca6b071511d633709cf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4126d927aac74375b2d65068c2420061 in term 1.
I20260812 02:39:29.665892 23814 leader_election.cc:302] T 0a6d4acbe87745608b0b12577d445f5b P 4126d927aac74375b2d65068c2420061 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20b9d93915064ca6b071511d633709cf, 4126d927aac74375b2d65068c2420061; no voters:
I20260812 02:39:29.666302 23991 raft_consensus.cc:2781] T 0a6d4acbe87745608b0b12577d445f5b P 4126d927aac74375b2d65068c2420061 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:39:29.666790 23991 raft_consensus.cc:686] T 0a6d4acbe87745608b0b12577d445f5b P 4126d927aac74375b2d65068c2420061 [term 1 LEADER]: Becoming Leader. State: Replica: 4126d927aac74375b2d65068c2420061, State: Running, Role: LEADER
I20260812 02:39:29.667258 23991 consensus_queue.cc:237] T 0a6d4acbe87745608b0b12577d445f5b P 4126d927aac74375b2d65068c2420061 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c97d93336a7454db0e8720dfd306be0" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 46179 } } peers { permanent_uuid: "20b9d93915064ca6b071511d633709cf" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37475 } } peers { permanent_uuid: "4126d927aac74375b2d65068c2420061" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 41889 } }
I20260812 02:39:29.672353 23387 catalog_manager.cc:5495] T 0a6d4acbe87745608b0b12577d445f5b P 4126d927aac74375b2d65068c2420061 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4126d927aac74375b2d65068c2420061 (127.7.132.131). New cstate: current_term: 1 leader_uuid: "4126d927aac74375b2d65068c2420061" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c97d93336a7454db0e8720dfd306be0" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 46179 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20b9d93915064ca6b071511d633709cf" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4126d927aac74375b2d65068c2420061" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 41889 } health_report { overall_health: HEALTHY } } }
I20260812 02:39:29.698510 23981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:29.692986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56352 (local address 127.7.132.131:41889)
0812 02:39:29.693216 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0812 02:39:29.693221 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:39:29.693237 (+ 16us) server_negotiation.cc:407] Connection header received
0812 02:39:29.693380 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:39:29.693384 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:39:29.693465 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:39:29.693608 (+ 143us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 02:39:29.693974 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:29.694922 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:29.696312 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:29.696733 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:29.696838 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:39:29.697308 (+ 470us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 02:39:29.697834 (+ 526us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 02:39:29.697974 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0812 02:39:29.698043 (+ 69us) server_negotiation.cc:299] Negotiation successful
0812 02:39:29.698191 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 02:39:29.707562 23739 raft_consensus.cc:1261] T 0a6d4acbe87745608b0b12577d445f5b P 20b9d93915064ca6b071511d633709cf [term 1 FOLLOWER]: Refusing update from remote peer 4126d927aac74375b2d65068c2420061: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:39:29.707541 23567 raft_consensus.cc:1261] T 0a6d4acbe87745608b0b12577d445f5b P 7c97d93336a7454db0e8720dfd306be0 [term 1 FOLLOWER]: Refusing update from remote peer 4126d927aac74375b2d65068c2420061: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260812 02:39:29.707962 23790 tablet.cc:2369] T 0a6d4acbe87745608b0b12577d445f5b P 20b9d93915064ca6b071511d633709cf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 02:39:29.708674 23994 consensus_queue.cc:1035] T 0a6d4acbe87745608b0b12577d445f5b P 4126d927aac74375b2d65068c2420061 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20b9d93915064ca6b071511d633709cf" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:39:29.709028 23991 consensus_queue.cc:1035] T 0a6d4acbe87745608b0b12577d445f5b P 4126d927aac74375b2d65068c2420061 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c97d93336a7454db0e8720dfd306be0" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 46179 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:39:29.720407 23997 mvcc.cc:205] Tried to move back new op lower bound from 7317513706311168000 to 7317513706161033216. Current Snapshot: MvccSnapshot[applied={T|T < 7317513706311168000}]
I20260812 02:39:29.721666 23998 mvcc.cc:205] Tried to move back new op lower bound from 7317513706311168000 to 7317513706161033216. Current Snapshot: MvccSnapshot[applied={T|T < 7317513706311168000}]
I20260812 02:39:29.726259 24000 mvcc.cc:205] Tried to move back new op lower bound from 7317513706311168000 to 7317513706161033216. Current Snapshot: MvccSnapshot[applied={T|T < 7317513706311168000}]
I20260812 02:39:29.738969 23384 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34460:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 02:39:29.739277 23384 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34460:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 02:39:29.742997 23384 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 279e011032c94987b4f63d0ad2772a40: Sending DeleteTablet for 3 replicas of tablet 0a6d4acbe87745608b0b12577d445f5b
I20260812 02:39:29.744316 23545 tablet_service.cc:1514] Processing DeleteTablet for tablet 0a6d4acbe87745608b0b12577d445f5b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:39:29 UTC) from {username='kudu', principal='kudu/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:36160
I20260812 02:39:29.744323 23871 tablet_service.cc:1514] Processing DeleteTablet for tablet 0a6d4acbe87745608b0b12577d445f5b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:39:29 UTC) from {username='kudu', principal='kudu/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:56348
I20260812 02:39:29.747584 23715 tablet_service.cc:1514] Processing DeleteTablet for tablet 0a6d4acbe87745608b0b12577d445f5b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:39:29 UTC) from {username='kudu', principal='kudu/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:55350
I20260812 02:39:29.749645 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 23429
I20260812 02:39:29.772425 23979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:29.750655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36174 (local address 127.7.132.129:46179)
0812 02:39:29.750853 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0812 02:39:29.750859 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:39:29.750879 (+ 20us) server_negotiation.cc:407] Connection header received
0812 02:39:29.751111 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:39:29.751117 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:39:29.751205 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:39:29.751305 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:39:29.755442 (+ 4137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:29.756659 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:29.758733 (+ 2074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:29.759433 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:29.759571 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:39:29.763603 (+ 4032us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:39:29.763621 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:39:29.763640 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:39:29.763696 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:39:29.767849 (+ 4153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:39:29.768774 (+ 925us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:39:29.768784 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:39:29.768788 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:39:29.768977 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:39:29.769470 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:39:29.769475 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:39:29.769478 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:39:29.770045 (+ 567us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:39:29.770388 (+ 343us) server_negotiation.cc:1035] Waiting for connection context
0812 02:39:29.771739 (+ 1351us) server_negotiation.cc:299] Negotiation successful
0812 02:39:29.771993 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260812 02:39:29.773648 23615 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:39:29.775699 24011 tablet_replica.cc:330] T 0a6d4acbe87745608b0b12577d445f5b P 20b9d93915064ca6b071511d633709cf: stopping tablet replica
I20260812 02:39:29.776336 24011 raft_consensus.cc:2229] T 0a6d4acbe87745608b0b12577d445f5b P 20b9d93915064ca6b071511d633709cf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 02:39:29.777029 24011 raft_consensus.cc:2258] T 0a6d4acbe87745608b0b12577d445f5b P 20b9d93915064ca6b071511d633709cf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:39:29.785656 24011 ts_tablet_manager.cc:1905] T 0a6d4acbe87745608b0b12577d445f5b P 20b9d93915064ca6b071511d633709cf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:39:29.788196 24009 tablet_replica.cc:330] T 0a6d4acbe87745608b0b12577d445f5b P 4126d927aac74375b2d65068c2420061: stopping tablet replica
I20260812 02:39:29.789120 24009 raft_consensus.cc:2229] T 0a6d4acbe87745608b0b12577d445f5b P 4126d927aac74375b2d65068c2420061 [term 1 LEADER]: Raft consensus shutting down.
I20260812 02:39:29.790050 24009 raft_consensus.cc:2258] T 0a6d4acbe87745608b0b12577d445f5b P 4126d927aac74375b2d65068c2420061 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:39:29.796923 24009 ts_tablet_manager.cc:1905] T 0a6d4acbe87745608b0b12577d445f5b P 4126d927aac74375b2d65068c2420061: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:39:29.797030 24011 ts_tablet_manager.cc:1918] T 0a6d4acbe87745608b0b12577d445f5b P 20b9d93915064ca6b071511d633709cf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 02:39:29.797286 24011 log.cc:1196] T 0a6d4acbe87745608b0b12577d445f5b P 20b9d93915064ca6b071511d633709cf: Deleting WAL directory at /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/ts-1/wal/wals/0a6d4acbe87745608b0b12577d445f5b
I20260812 02:39:29.797923 24011 ts_tablet_manager.cc:1939] T 0a6d4acbe87745608b0b12577d445f5b P 20b9d93915064ca6b071511d633709cf: Deleting consensus metadata
I20260812 02:39:29.800246 23370 catalog_manager.cc:4841] TS 20b9d93915064ca6b071511d633709cf (127.7.132.130:37475): tablet 0a6d4acbe87745608b0b12577d445f5b (table test-table [id=de9feaac0edc4f8985afd848155ab93a]) successfully deleted
I20260812 02:39:29.805081 24009 ts_tablet_manager.cc:1918] T 0a6d4acbe87745608b0b12577d445f5b P 4126d927aac74375b2d65068c2420061: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 02:39:29.805274 24009 log.cc:1196] T 0a6d4acbe87745608b0b12577d445f5b P 4126d927aac74375b2d65068c2420061: Deleting WAL directory at /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786502156971023-7698-0/minicluster-data/ts-2/wal/wals/0a6d4acbe87745608b0b12577d445f5b
I20260812 02:39:29.805866 24009 ts_tablet_manager.cc:1939] T 0a6d4acbe87745608b0b12577d445f5b P 4126d927aac74375b2d65068c2420061: Deleting consensus metadata
I20260812 02:39:29.807768 23371 catalog_manager.cc:4841] TS 4126d927aac74375b2d65068c2420061 (127.7.132.131:41889): tablet 0a6d4acbe87745608b0b12577d445f5b (table test-table [id=de9feaac0edc4f8985afd848155ab93a]) successfully deleted
I20260812 02:39:29.880992 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 23429
I20260812 02:39:29.881070 24010 tablet_replica.cc:330] T 0a6d4acbe87745608b0b12577d445f5b P 7c97d93336a7454db0e8720dfd306be0: stopping tablet replica
W20260812 02:39:29.910776 23372 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.132.129:46179 (error 108)
I20260812 02:39:29.910887 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 23627
I20260812 02:39:29.912833 23977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:29.912353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:46179 (local address 127.0.0.1:36184)
0812 02:39:29.912531 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0812 02:39:29.912653 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.129:46179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260812 02:39:29.913098 23372 catalog_manager.cc:4584] TS 7c97d93336a7454db0e8720dfd306be0 (127.7.132.129:46179): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0a6d4acbe87745608b0b12577d445f5b: Network error: Client connection negotiation failed: client connection to 127.7.132.129:46179: connect: Connection refused (error 111)
I20260812 02:39:29.927426 23978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:29.911814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55352 (local address 127.7.132.130:37475)
0812 02:39:29.912001 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0812 02:39:29.912008 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:39:29.912115 (+ 107us) server_negotiation.cc:407] Connection header received
0812 02:39:29.912310 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:39:29.912316 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:39:29.912402 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:39:29.912540 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:39:29.914204 (+ 1664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:29.915354 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:29.916634 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:29.917289 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:29.917426 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:39:29.920146 (+ 2720us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:39:29.920166 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:39:29.920187 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:39:29.920233 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:39:29.924228 (+ 3995us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:39:29.925002 (+ 774us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:39:29.925012 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:39:29.925018 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:39:29.925189 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:39:29.925573 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:39:29.925579 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:39:29.925583 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:39:29.926223 (+ 640us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:39:29.926572 (+ 349us) server_negotiation.cc:1035] Waiting for connection context
0812 02:39:29.926784 (+ 212us) server_negotiation.cc:299] Negotiation successful
0812 02:39:29.927019 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 02:39:29.928340 23784 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:39:29.966900 23977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:29.966304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:46179 (local address 127.0.0.1:36186)
0812 02:39:29.966563 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0812 02:39:29.966691 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.129:46179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260812 02:39:29.968160 23977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:29.967735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:46179 (local address 127.0.0.1:36196)
0812 02:39:29.967914 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0812 02:39:29.968006 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.129:46179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260812 02:39:30.021476 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 23627
I20260812 02:39:30.028985 23977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:30.028539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:46179 (local address 127.0.0.1:36206)
0812 02:39:30.028748 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0812 02:39:30.028829 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.129:46179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260812 02:39:30.030068 23977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:30.029720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:46179 (local address 127.0.0.1:36212)
0812 02:39:30.029875 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0812 02:39:30.029953 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.129:46179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260812 02:39:30.051824 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 23793
I20260812 02:39:30.069756 23981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:30.052659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56366 (local address 127.7.132.131:41889)
0812 02:39:30.052849 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0812 02:39:30.052854 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:39:30.053280 (+ 426us) server_negotiation.cc:407] Connection header received
0812 02:39:30.053361 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:39:30.053366 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:39:30.053484 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:39:30.053592 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:39:30.055216 (+ 1624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:30.056522 (+ 1306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:30.057954 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:30.058728 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:30.058885 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:39:30.061693 (+ 2808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:39:30.061709 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:39:30.061729 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:39:30.061781 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:39:30.066469 (+ 4688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:39:30.067279 (+ 810us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:39:30.067290 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:39:30.067293 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:39:30.067514 (+ 221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:39:30.067885 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:39:30.067891 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:39:30.067894 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:39:30.068458 (+ 564us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:39:30.068804 (+ 346us) server_negotiation.cc:1035] Waiting for connection context
0812 02:39:30.069079 (+ 275us) server_negotiation.cc:299] Negotiation successful
0812 02:39:30.069320 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 02:39:30.070672 23952 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:39:30.110975 23977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:30.110390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:46179 (local address 127.0.0.1:36218)
0812 02:39:30.110636 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0812 02:39:30.110755 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.129:46179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260812 02:39:30.112146 23977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:30.111687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:46179 (local address 127.0.0.1:36226)
0812 02:39:30.111911 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0812 02:39:30.112003 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.129:46179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114}
I20260812 02:39:30.195629 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 23793
I20260812 02:39:30.224648 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 23350
I20260812 02:39:30.225955 23421 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:39:30.340793 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 23350
I20260812 02:39:30.373129 7698 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260812 02:39:30.718744 7698 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260812 02:39:30.718958 7698 mini_ranger.cc:66] Stopping Ranger...
I20260812 02:39:31.064137 7698 mini_ranger.cc:68] Stopped Ranger
2026-08-12 02:39:31.064 UTC [22207] LOG: received smart shutdown request
2026-08-12 02:39:31.067 UTC [22207] LOG: background worker "logical replication launcher" (PID 22216) exited with exit code 1
2026-08-12 02:39:31.068 UTC [22211] LOG: shutting down
2026-08-12 02:39:31.244 UTC [22207] LOG: database system is shut down
2026-08-12T02:39:31Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (55509 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502156971023-7698-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 02:39:31 dist-test-slave-mn4r krb5kdc[24030](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 02:39:31 dist-test-slave-mn4r krb5kdc[24030](info): set up 2 sockets
Aug 12 02:39:31 dist-test-slave-mn4r krb5kdc[24030](info): commencing operation
krb5kdc: starting...
W20260812 02:39:32.600131 7698 mini_kdc.cc:121] Time spent starting KDC: real 1.181s user 0.000s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502156971023-7698-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502156971023-7698-0/krb5kdc/test-user.keytab.
Aug 12 02:39:32 dist-test-slave-mn4r krb5kdc[24030](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502372, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T02:39:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T02:39:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.132.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
I20260812 02:39:32.881853 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502156971023-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502156971023-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502156971023-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:44813
--webserver_interface=127.7.132.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:45773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.132.190:44813,127.7.132.189:46569,127.7.132.188:42281
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:39:33.225756 24046 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:39:33.226054 24046 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:39:33.226276 24046 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:39:33.233531 24046 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:39:33.233639 24046 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:39:33.233727 24046 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:39:33.233839 24046 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:39:33.233903 24046 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:39:33.244048 24046 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:45773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502156971023-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502156971023-7698-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.132.190:44813,127.7.132.189:46569,127.7.132.188:42281
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:44813
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.132.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502156971023-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:39:33.245437 24046 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:39:33.247013 24046 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:39:33.259151 24052 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:39:33.259209 24053 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:39:33.259181 24056 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:39:33.260390 24046 server_base.cc:1034] running on GCE node
I20260812 02:39:33.261197 24046 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:39:33.262751 24046 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:39:33.264019 24046 hybrid_clock.cc:648] HybridClock initialized: now 1786502373263978 us; error 46 us; skew 500 ppm
I20260812 02:39:33.264392 24046 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:39:33 dist-test-slave-mn4r krb5kdc[24030](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502373, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:39:33.267809 24046 init.cc:377] Logged in from keytab as kudu/127.7.132.190@KRBTEST.COM (short username kudu)
I20260812 02:39:33.269384 24046 webserver.cc:460] Webserver started at http://127.7.132.190:39215/ using document root <none> and password file <none>
I20260812 02:39:33.270179 24046 fs_manager.cc:359] Metadata directory not provided
I20260812 02:39:33.270330 24046 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:39:33.270665 24046 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:39:33.273828 24046 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502156971023-7698-0/minicluster-data/master-0/data/instance:
uuid: "e6f0690f6e2c43aa849459cd688f58f5"
format_stamp: "Formatted at 2026-08-12 02:39:33 on dist-test-slave-mn4r"
server_key: "0e684533c4af893e39b5f7c8fe2fb6e3"
server_key_iv: "0a5df06f771fb2936595b589e7166382"
server_key_version: "encryptionkey@0"
I20260812 02:39:33.274793 24046 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502156971023-7698-0/minicluster-data/master-0/wal/instance:
uuid: "e6f0690f6e2c43aa849459cd688f58f5"
format_stamp: "Formatted at 2026-08-12 02:39:33 on dist-test-slave-mn4r"
server_key: "0e684533c4af893e39b5f7c8fe2fb6e3"
server_key_iv: "0a5df06f771fb2936595b589e7166382"
server_key_version: "encryptionkey@0"
I20260812 02:39:33.280287 24046 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 02:39:33.284061 24064 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:39:33.285488 24046 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 02:39:33.285710 24046 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502156971023-7698-0/minicluster-data/master-0/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502156971023-7698-0/minicluster-data/master-0/wal
uuid: "e6f0690f6e2c43aa849459cd688f58f5"
format_stamp: "Formatted at 2026-08-12 02:39:33 on dist-test-slave-mn4r"
server_key: "0e684533c4af893e39b5f7c8fe2fb6e3"
server_key_iv: "0a5df06f771fb2936595b589e7166382"
server_key_version: "encryptionkey@0"
I20260812 02:39:33.285923 24046 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502156971023-7698-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502156971023-7698-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502156971023-7698-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:39:33.305101 24046 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:39:33.309290 24046 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:39:33.309594 24046 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:39:33.328610 24046 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.190:44813
I20260812 02:39:33.328658 24124 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.190:44813 every 8 connection(s)
I20260812 02:39:33.330050 24046 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
I20260812 02:39:33.333849 24125 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:39:33.339608 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 24046
I20260812 02:39:33.339857 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502156971023-7698-0/minicluster-data/master-0/wal/instance
I20260812 02:39:33.340727 7698 external_mini_cluster.cc:1428] Setting key 24426f19ee85a314139fdde2d4059cc9
I20260812 02:39:33.341049 24125 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.132.190" port: 44813 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 02:39:33 dist-test-slave-mn4r krb5kdc[24030](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502373, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.190@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.132.189@KRBTEST.COM; defaulting to no policy
I20260812 02:39:33.379281 24127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:33.345268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:44813 (local address 127.0.0.1:45478)
0812 02:39:33.355489 (+ 10221us) negotiation.cc:107] Waiting for socket to connect
0812 02:39:33.355551 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0812 02:39:33.358518 (+ 2967us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:39:33.359686 (+ 1168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:39:33.359696 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:39:33.360586 (+ 890us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:39:33.361128 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:39:33.361158 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:39:33.363652 (+ 2494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:39:33.363659 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:39:33.365747 (+ 2088us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:39:33.365764 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:39:33.367010 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:39:33.367017 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:39:33.367252 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:39:33.368050 (+ 798us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:39:33.368081 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:39:33.370975 (+ 2894us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:39:33.374535 (+ 3560us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:39:33.374549 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:39:33.374562 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:39:33.374966 (+ 404us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:39:33.375445 (+ 479us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:39:33.375451 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:39:33.375456 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:39:33.375625 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:39:33.376625 (+ 1000us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:39:33.376634 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:39:33.376851 (+ 217us) client_negotiation.cc:769] Sending connection context
0812 02:39:33.377182 (+ 331us) client_negotiation.cc:240] Negotiation successful
0812 02:39:33.377799 (+ 617us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9914,"mutex_wait_us":181,"thread_start_us":9573,"threads_started":1}
I20260812 02:39:33.379299 24128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:33.347088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45478 (local address 127.7.132.190:44813)
0812 02:39:33.355456 (+ 8368us) server_negotiation.cc:206] Beginning negotiation
0812 02:39:33.355476 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0812 02:39:33.356110 (+ 634us) server_negotiation.cc:407] Connection header received
0812 02:39:33.359041 (+ 2931us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:39:33.359052 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:39:33.359375 (+ 323us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:39:33.359582 (+ 207us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:39:33.361372 (+ 1790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:33.363455 (+ 2083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:33.365901 (+ 2446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:33.366745 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:33.366962 (+ 217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:39:33.371221 (+ 4259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:39:33.371230 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:39:33.371244 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:39:33.371273 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:39:33.374404 (+ 3131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:39:33.375109 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:39:33.375117 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:39:33.375121 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:39:33.375331 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:39:33.375722 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:39:33.375728 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:39:33.375733 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:39:33.376388 (+ 655us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:39:33.376564 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0812 02:39:33.377298 (+ 734us) server_negotiation.cc:299] Negotiation successful
0812 02:39:33.377690 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":54,"server-negotiator.queue_time_us":8051,"thread_start_us":8934,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.189@KRBTEST.COM" created.
I20260812 02:39:33.383224 24125 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.132.189" port: 46569 } has no permanent_uuid. Determining permanent_uuid...
I20260812 02:39:33.384371 24127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:33.383926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.189:46569 (local address 127.0.0.1:45508)
0812 02:39:33.384121 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0812 02:39:33.384242 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.189:46569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
W20260812 02:39:33.384698 24066 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.132.189:46569: connect: Connection refused (error 111)
I20260812 02:39:33.386585 24127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:33.386188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.189:46569 (local address 127.0.0.1:45512)
0812 02:39:33.386351 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0812 02:39:33.386433 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.189:46569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260812 02:39:33.386855 24125 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.132.189:46569: Network error: Client connection negotiation failed: client connection to 127.7.132.189:46569: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.189.keytab.
Entry for principal kudu/127.7.132.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.189.keytab.
I20260812 02:39:33.413723 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502156971023-7698-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502156971023-7698-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502156971023-7698-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502156971023-7698-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.189:46569
--webserver_interface=127.7.132.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.189.keytab
--principal=kudu/127.7.132.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:45773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.132.190:44813,127.7.132.189:46569,127.7.132.188:42281
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 02:39:33.436378 24125 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.132.189" port: 46569 } attempt: 1
I20260812 02:39:33.438163 24127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:33.437549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.189:46569 (local address 127.0.0.1:45528)
0812 02:39:33.437807 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0812 02:39:33.437937 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.189:46569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
I20260812 02:39:33.439349 24127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:33.438981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.189:46569 (local address 127.0.0.1:45538)
0812 02:39:33.439139 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0812 02:39:33.439208 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.189:46569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
W20260812 02:39:33.439653 24125 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.132.189:46569: Network error: Client connection negotiation failed: client connection to 127.7.132.189:46569: connect: Connection refused (error 111)
I20260812 02:39:33.507899 24125 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.132.189" port: 46569 } attempt: 2
I20260812 02:39:33.509289 24127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:33.508789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.189:46569 (local address 127.0.0.1:45544)
0812 02:39:33.509049 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0812 02:39:33.509141 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.189:46569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260812 02:39:33.510423 24127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:33.509967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.189:46569 (local address 127.0.0.1:45560)
0812 02:39:33.510189 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0812 02:39:33.510262 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.189:46569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
W20260812 02:39:33.510648 24125 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.132.189:46569: Network error: Client connection negotiation failed: client connection to 127.7.132.189:46569: connect: Connection refused (error 111)
I20260812 02:39:33.601907 24125 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.132.189" port: 46569 } attempt: 3
I20260812 02:39:33.603084 24127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:33.602632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.189:46569 (local address 127.0.0.1:45570)
0812 02:39:33.602834 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0812 02:39:33.602952 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.189:46569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260812 02:39:33.604096 24127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:33.603742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.189:46569 (local address 127.0.0.1:45582)
0812 02:39:33.603891 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0812 02:39:33.603967 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.189:46569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260812 02:39:33.604285 24125 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.132.189:46569: Network error: Client connection negotiation failed: client connection to 127.7.132.189:46569: connect: Connection refused (error 111)
I20260812 02:39:33.747556 24125 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.132.189" port: 46569 } attempt: 4
I20260812 02:39:33.748790 24127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:33.748239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.189:46569 (local address 127.0.0.1:45594)
0812 02:39:33.748571 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0812 02:39:33.748655 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.189:46569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158}
I20260812 02:39:33.749837 24127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:33.749474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.189:46569 (local address 127.0.0.1:45602)
0812 02:39:33.749639 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0812 02:39:33.749723 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.189:46569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260812 02:39:33.750010 24125 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.132.189:46569: Network error: Client connection negotiation failed: client connection to 127.7.132.189:46569: connect: Connection refused (error 111)
W20260812 02:39:33.758699 24132 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:39:33.759030 24132 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:39:33.759188 24132 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:39:33.766304 24132 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:39:33.766443 24132 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:39:33.766554 24132 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:39:33.766664 24132 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:39:33.766752 24132 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:39:33.776647 24132 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:45773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502156971023-7698-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502156971023-7698-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.7.132.190:44813,127.7.132.189:46569,127.7.132.188:42281
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.189.keytab
--principal=kudu/127.7.132.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.189:46569
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502156971023-7698-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.7.132.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502156971023-7698-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:39:33.777995 24132 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:39:33.779358 24132 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:39:33.791980 24137 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:39:33.791927 24138 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:39:33.791939 24140 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:39:33.792944 24132 server_base.cc:1034] running on GCE node
I20260812 02:39:33.793767 24132 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:39:33.795264 24132 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:39:33.796474 24132 hybrid_clock.cc:648] HybridClock initialized: now 1786502373796420 us; error 43 us; skew 500 ppm
I20260812 02:39:33.796826 24132 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:39:33 dist-test-slave-mn4r krb5kdc[24030](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502373, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:39:33.800081 24132 init.cc:377] Logged in from keytab as kudu/127.7.132.189@KRBTEST.COM (short username kudu)
I20260812 02:39:33.801586 24132 webserver.cc:460] Webserver started at http://127.7.132.189:43809/ using document root <none> and password file <none>
I20260812 02:39:33.802380 24132 fs_manager.cc:359] Metadata directory not provided
I20260812 02:39:33.802529 24132 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:39:33.802841 24132 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:39:33.805817 24132 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502156971023-7698-0/minicluster-data/master-1/data/instance:
uuid: "1389b9e0e56541e1990cf226310c2a87"
format_stamp: "Formatted at 2026-08-12 02:39:33 on dist-test-slave-mn4r"
server_key: "c6038124148190d1c156628375a0cd9a"
server_key_iv: "78a5a31a96fc6555795c3e282f45abbf"
server_key_version: "encryptionkey@0"
I20260812 02:39:33.806702 24132 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502156971023-7698-0/minicluster-data/master-1/wal/instance:
uuid: "1389b9e0e56541e1990cf226310c2a87"
format_stamp: "Formatted at 2026-08-12 02:39:33 on dist-test-slave-mn4r"
server_key: "c6038124148190d1c156628375a0cd9a"
server_key_iv: "78a5a31a96fc6555795c3e282f45abbf"
server_key_version: "encryptionkey@0"
I20260812 02:39:33.811896 24132 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260812 02:39:33.815471 24151 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:39:33.816865 24132 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 02:39:33.817068 24132 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502156971023-7698-0/minicluster-data/master-1/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502156971023-7698-0/minicluster-data/master-1/wal
uuid: "1389b9e0e56541e1990cf226310c2a87"
format_stamp: "Formatted at 2026-08-12 02:39:33 on dist-test-slave-mn4r"
server_key: "c6038124148190d1c156628375a0cd9a"
server_key_iv: "78a5a31a96fc6555795c3e282f45abbf"
server_key_version: "encryptionkey@0"
I20260812 02:39:33.817268 24132 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502156971023-7698-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502156971023-7698-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502156971023-7698-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:39:33.839298 24132 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:39:33.843484 24132 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:39:33.843789 24132 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:39:33.862352 24132 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.189:46569
I20260812 02:39:33.862385 24209 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.189:46569 every 8 connection(s)
I20260812 02:39:33.863713 24132 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502156971023-7698-0/minicluster-data/master-1/data/info.pb
I20260812 02:39:33.867388 24211 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:39:33.871155 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 24132
I20260812 02:39:33.871364 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502156971023-7698-0/minicluster-data/master-1/wal/instance
I20260812 02:39:33.871933 7698 external_mini_cluster.cc:1428] Setting key ec29ab0e3eabbafbeb7c48a95f8ae7b0
I20260812 02:39:33.875658 24211 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.132.190" port: 44813 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 02:39:33 dist-test-slave-mn4r krb5kdc[24030](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502373, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.189@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 02:39:33.901693 24128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:33.879135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45490 (local address 127.7.132.190:44813)
0812 02:39:33.879410 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0812 02:39:33.879418 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:39:33.880652 (+ 1234us) server_negotiation.cc:407] Connection header received
0812 02:39:33.882709 (+ 2057us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:39:33.882718 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:39:33.882818 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:39:33.882952 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:39:33.885460 (+ 2508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:33.886214 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:33.889001 (+ 2787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:33.889434 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:33.889605 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:39:33.895026 (+ 5421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:39:33.895033 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:39:33.895037 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:39:33.895071 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:39:33.897950 (+ 2879us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:39:33.899120 (+ 1170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:39:33.899125 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:39:33.899127 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:39:33.899245 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:39:33.899830 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:39:33.899833 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:39:33.899835 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:39:33.900120 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:39:33.900233 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0812 02:39:33.901270 (+ 1037us) server_negotiation.cc:299] Negotiation successful
0812 02:39:33.901456 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
WARNING: no policy specified for kudu/127.7.132.188@KRBTEST.COM; defaulting to no policy
I20260812 02:39:33.903605 24213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:33.879673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:44813 (local address 127.0.0.1:45490)
0812 02:39:33.880472 (+ 799us) negotiation.cc:107] Waiting for socket to connect
0812 02:39:33.880518 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 02:39:33.882430 (+ 1912us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:39:33.883318 (+ 888us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:39:33.883333 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:39:33.884292 (+ 959us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:39:33.885246 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:39:33.885278 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:39:33.886386 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:39:33.886395 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:39:33.888817 (+ 2422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:39:33.888839 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:39:33.889715 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:39:33.889726 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:39:33.890155 (+ 429us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:39:33.891014 (+ 859us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:39:33.891057 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:39:33.894755 (+ 3698us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:39:33.898174 (+ 3419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:39:33.898183 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:39:33.898203 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:39:33.898984 (+ 781us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:39:33.899384 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:39:33.899391 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:39:33.899397 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:39:33.899701 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:39:33.900265 (+ 564us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:39:33.900277 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:39:33.900805 (+ 528us) client_negotiation.cc:769] Sending connection context
0812 02:39:33.901194 (+ 389us) client_negotiation.cc:240] Negotiation successful
0812 02:39:33.901781 (+ 587us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":536,"thread_start_us":354,"threads_started":1}
I20260812 02:39:33.905534 24211 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.132.189" port: 46569 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.188@KRBTEST.COM" created.
Aug 12 02:39:33 dist-test-slave-mn4r krb5kdc[24030](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502373, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.189@KRBTEST.COM for kudu/127.7.132.189@KRBTEST.COM
I20260812 02:39:33.922340 24213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:33.906340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.189:46569 (local address 127.0.0.1:45612)
0812 02:39:33.906609 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0812 02:39:33.906633 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0812 02:39:33.906742 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:39:33.908036 (+ 1294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:39:33.908041 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:39:33.908551 (+ 510us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:39:33.908703 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:39:33.908712 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:39:33.910275 (+ 1563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:39:33.910283 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:39:33.911217 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:39:33.911229 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:39:33.912170 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:39:33.912177 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:39:33.912398 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:39:33.913085 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:39:33.913103 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:39:33.915408 (+ 2305us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:39:33.919466 (+ 4058us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:39:33.919471 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:39:33.919476 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:39:33.919893 (+ 417us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:39:33.920308 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:39:33.920315 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:39:33.920319 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:39:33.920505 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:39:33.921216 (+ 711us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:39:33.921224 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:39:33.921379 (+ 155us) client_negotiation.cc:769] Sending connection context
0812 02:39:33.921544 (+ 165us) client_negotiation.cc:240] Negotiation successful
0812 02:39:33.921862 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100}
I20260812 02:39:33.922511 24216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:33.906602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45612 (local address 127.7.132.189:46569)
0812 02:39:33.907538 (+ 936us) server_negotiation.cc:206] Beginning negotiation
0812 02:39:33.907545 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:39:33.907574 (+ 29us) server_negotiation.cc:407] Connection header received
0812 02:39:33.907624 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:39:33.907631 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:39:33.907832 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:39:33.908047 (+ 215us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:39:33.908878 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:33.910070 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:33.911372 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:33.911974 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:33.912120 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:39:33.915640 (+ 3520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:39:33.915653 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:39:33.915667 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:39:33.915703 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:39:33.919281 (+ 3578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:39:33.920055 (+ 774us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:39:33.920063 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:39:33.920066 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:39:33.920188 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:39:33.920642 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:39:33.920649 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:39:33.920653 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:39:33.921058 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:39:33.921307 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0812 02:39:33.921793 (+ 486us) server_negotiation.cc:299] Negotiation successful
0812 02:39:33.922059 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":770,"thread_start_us":612,"threads_started":1}
I20260812 02:39:33.925251 24211 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.132.188" port: 42281 } has no permanent_uuid. Determining permanent_uuid...
I20260812 02:39:33.926574 24213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:33.926051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:42281 (local address 127.0.0.1:54256)
0812 02:39:33.926305 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0812 02:39:33.926416 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.188:42281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136}
W20260812 02:39:33.926863 24152 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.132.188:42281: connect: Connection refused (error 111)
I20260812 02:39:33.928465 24213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:33.928136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:42281 (local address 127.0.0.1:54264)
0812 02:39:33.928275 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0812 02:39:33.928360 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.188:42281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260812 02:39:33.928763 24211 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.132.188:42281: Network error: Client connection negotiation failed: client connection to 127.7.132.188:42281: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.188.keytab.
Entry for principal kudu/127.7.132.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.188.keytab.
I20260812 02:39:33.942169 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502156971023-7698-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502156971023-7698-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502156971023-7698-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502156971023-7698-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.188:42281
--webserver_interface=127.7.132.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.188.keytab
--principal=kudu/127.7.132.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:45773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.132.190:44813,127.7.132.189:46569,127.7.132.188:42281
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 02:39:33.978188 24211 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.132.188" port: 42281 } attempt: 1
I20260812 02:39:33.979373 24213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:33.978898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:42281 (local address 127.0.0.1:54274)
0812 02:39:33.979115 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0812 02:39:33.979225 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.188:42281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260812 02:39:33.980347 24213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:33.980011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:42281 (local address 127.0.0.1:54282)
0812 02:39:33.980183 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0812 02:39:33.980247 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.188:42281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260812 02:39:33.980563 24211 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.132.188:42281: Network error: Client connection negotiation failed: client connection to 127.7.132.188:42281: connect: Connection refused (error 111)
I20260812 02:39:34.048803 24211 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.132.188" port: 42281 } attempt: 2
I20260812 02:39:34.049322 24125 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.132.189" port: 46569 } attempt: 5
I20260812 02:39:34.050177 24213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:34.049591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:42281 (local address 127.0.0.1:54296)
0812 02:39:34.049792 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0812 02:39:34.049916 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.188:42281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260812 02:39:34.052388 24213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:34.051950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:42281 (local address 127.0.0.1:54312)
0812 02:39:34.052138 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0812 02:39:34.052231 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.188:42281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260812 02:39:34.052711 24211 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.132.188:42281: Network error: Client connection negotiation failed: client connection to 127.7.132.188:42281: connect: Connection refused (error 111)
Aug 12 02:39:34 dist-test-slave-mn4r krb5kdc[24030](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502373, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.190@KRBTEST.COM for kudu/127.7.132.189@KRBTEST.COM
I20260812 02:39:34.064414 24127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:34.049981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.189:46569 (local address 127.0.0.1:45614)
0812 02:39:34.050269 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0812 02:39:34.050288 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 02:39:34.050429 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:39:34.050910 (+ 481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:39:34.050916 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:39:34.051343 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:39:34.051567 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:39:34.051578 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:39:34.052977 (+ 1399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:39:34.052983 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:39:34.053948 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:39:34.053960 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:39:34.054844 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:39:34.054850 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:39:34.055058 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:39:34.055755 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:39:34.055775 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:39:34.058304 (+ 2529us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:39:34.061578 (+ 3274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:39:34.061585 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:39:34.061589 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:39:34.062134 (+ 545us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:39:34.062566 (+ 432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:39:34.062571 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:39:34.062576 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:39:34.062734 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:39:34.063319 (+ 585us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:39:34.063324 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:39:34.063500 (+ 176us) client_negotiation.cc:769] Sending connection context
0812 02:39:34.063658 (+ 158us) client_negotiation.cc:240] Negotiation successful
0812 02:39:34.064030 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175}
I20260812 02:39:34.064409 24216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:34.050309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45614 (local address 127.7.132.189:46569)
0812 02:39:34.050473 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0812 02:39:34.050481 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:39:34.050531 (+ 50us) server_negotiation.cc:407] Connection header received
0812 02:39:34.050599 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:39:34.050606 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:39:34.050729 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:39:34.050857 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:39:34.051734 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:34.052798 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:34.054138 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:34.054646 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:34.054828 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:39:34.058515 (+ 3687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:39:34.058520 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:39:34.058524 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:39:34.058557 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:39:34.061417 (+ 2860us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:39:34.062285 (+ 868us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:39:34.062296 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:39:34.062298 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:39:34.062437 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:39:34.062876 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:39:34.062881 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:39:34.062883 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:39:34.063178 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:39:34.063358 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0812 02:39:34.063785 (+ 427us) server_negotiation.cc:299] Negotiation successful
0812 02:39:34.064031 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 02:39:34.065570 24125 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.132.188" port: 42281 } has no permanent_uuid. Determining permanent_uuid...
I20260812 02:39:34.066805 24127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:34.066403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:42281 (local address 127.0.0.1:54324)
0812 02:39:34.066589 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0812 02:39:34.066666 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.188:42281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260812 02:39:34.067880 24127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:34.067535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:42281 (local address 127.0.0.1:54330)
0812 02:39:34.067696 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0812 02:39:34.067783 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.188:42281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260812 02:39:34.068131 24125 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.132.188:42281: Network error: Client connection negotiation failed: client connection to 127.7.132.188:42281: connect: Connection refused (error 111)
I20260812 02:39:34.119390 24125 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.132.188" port: 42281 } attempt: 1
I20260812 02:39:34.120802 24127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:34.120128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:42281 (local address 127.0.0.1:54342)
0812 02:39:34.120359 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0812 02:39:34.120564 (+ 205us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.188:42281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260812 02:39:34.122156 24127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:34.121686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:42281 (local address 127.0.0.1:54344)
0812 02:39:34.121846 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0812 02:39:34.121991 (+ 145us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.188:42281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260812 02:39:34.122373 24125 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.132.188:42281: Network error: Client connection negotiation failed: client connection to 127.7.132.188:42281: connect: Connection refused (error 111)
I20260812 02:39:34.143981 24211 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.132.188" port: 42281 } attempt: 3
I20260812 02:39:34.145372 24213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:34.144724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:42281 (local address 127.0.0.1:54348)
0812 02:39:34.145007 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0812 02:39:34.145136 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.188:42281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260812 02:39:34.146720 24213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:34.146209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:42281 (local address 127.0.0.1:54360)
0812 02:39:34.146479 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0812 02:39:34.146574 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.188:42281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146}
W20260812 02:39:34.146965 24211 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.132.188:42281: Network error: Client connection negotiation failed: client connection to 127.7.132.188:42281: connect: Connection refused (error 111)
I20260812 02:39:34.190685 24125 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.132.188" port: 42281 } attempt: 2
I20260812 02:39:34.191980 24127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:34.191373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:42281 (local address 127.0.0.1:54364)
0812 02:39:34.191656 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0812 02:39:34.191753 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.188:42281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260812 02:39:34.193198 24127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:34.192800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:42281 (local address 127.0.0.1:54376)
0812 02:39:34.193008 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0812 02:39:34.193077 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.188:42281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260812 02:39:34.193477 24125 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.132.188:42281: Network error: Client connection negotiation failed: client connection to 127.7.132.188:42281: connect: Connection refused (error 111)
W20260812 02:39:34.284575 24219 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:39:34.284879 24219 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:39:34.285076 24219 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260812 02:39:34.290203 24211 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.132.188" port: 42281 } attempt: 4
I20260812 02:39:34.291505 24213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:34.290933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:42281 (local address 127.0.0.1:54392)
0812 02:39:34.291199 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0812 02:39:34.291309 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.188:42281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260812 02:39:34.292665 24213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:34.292285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:42281 (local address 127.0.0.1:54398)
0812 02:39:34.292446 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0812 02:39:34.292531 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.188:42281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260812 02:39:34.292966 24211 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.132.188:42281: Network error: Client connection negotiation failed: client connection to 127.7.132.188:42281: connect: Connection refused (error 111)
W20260812 02:39:34.293844 24219 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:39:34.293982 24219 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:39:34.294123 24219 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:39:34.294232 24219 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:39:34.294317 24219 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:39:34.299731 24125 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.132.188" port: 42281 } attempt: 3
I20260812 02:39:34.300879 24127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:34.300431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:42281 (local address 127.0.0.1:54414)
0812 02:39:34.300623 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0812 02:39:34.300715 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.188:42281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260812 02:39:34.302026 24127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:34.301596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:42281 (local address 127.0.0.1:54424)
0812 02:39:34.301829 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0812 02:39:34.301922 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.188:42281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110}
W20260812 02:39:34.302314 24125 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.132.188:42281: Network error: Client connection negotiation failed: client connection to 127.7.132.188:42281: connect: Connection refused (error 111)
I20260812 02:39:34.305202 24219 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:45773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502156971023-7698-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502156971023-7698-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.7.132.190:44813,127.7.132.189:46569,127.7.132.188:42281
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.188.keytab
--principal=kudu/127.7.132.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.188:42281
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502156971023-7698-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.7.132.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502156971023-7698-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:39:34.306734 24219 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:39:34.308154 24219 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:39:34.320910 24227 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:39:34.321040 24230 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:39:34.320976 24224 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:39:34.327214 24219 server_base.cc:1034] running on GCE node
I20260812 02:39:34.328182 24219 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:39:34.329936 24219 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:39:34.331171 24219 hybrid_clock.cc:648] HybridClock initialized: now 1786502374331106 us; error 58 us; skew 500 ppm
I20260812 02:39:34.331568 24219 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:39:34 dist-test-slave-mn4r krb5kdc[24030](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502374, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:39:34.335455 24219 init.cc:377] Logged in from keytab as kudu/127.7.132.188@KRBTEST.COM (short username kudu)
I20260812 02:39:34.337229 24219 webserver.cc:460] Webserver started at http://127.7.132.188:38421/ using document root <none> and password file <none>
I20260812 02:39:34.338007 24219 fs_manager.cc:359] Metadata directory not provided
I20260812 02:39:34.338213 24219 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:39:34.338582 24219 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:39:34.341742 24219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502156971023-7698-0/minicluster-data/master-2/data/instance:
uuid: "3988c7c9497e439eb4f1e35439519f45"
format_stamp: "Formatted at 2026-08-12 02:39:34 on dist-test-slave-mn4r"
server_key: "24079ce04af6b8b513cad9859a87a72e"
server_key_iv: "d871bbc1b204ef9bdab7db3ef61343d1"
server_key_version: "encryptionkey@0"
I20260812 02:39:34.342725 24219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502156971023-7698-0/minicluster-data/master-2/wal/instance:
uuid: "3988c7c9497e439eb4f1e35439519f45"
format_stamp: "Formatted at 2026-08-12 02:39:34 on dist-test-slave-mn4r"
server_key: "24079ce04af6b8b513cad9859a87a72e"
server_key_iv: "d871bbc1b204ef9bdab7db3ef61343d1"
server_key_version: "encryptionkey@0"
I20260812 02:39:34.348308 24219 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 02:39:34.351768 24238 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:39:34.353055 24219 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 02:39:34.353247 24219 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502156971023-7698-0/minicluster-data/master-2/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502156971023-7698-0/minicluster-data/master-2/wal
uuid: "3988c7c9497e439eb4f1e35439519f45"
format_stamp: "Formatted at 2026-08-12 02:39:34 on dist-test-slave-mn4r"
server_key: "24079ce04af6b8b513cad9859a87a72e"
server_key_iv: "d871bbc1b204ef9bdab7db3ef61343d1"
server_key_version: "encryptionkey@0"
I20260812 02:39:34.353452 24219 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502156971023-7698-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502156971023-7698-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502156971023-7698-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:39:34.367137 24219 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:39:34.371488 24219 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:39:34.371788 24219 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:39:34.389991 24219 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.188:42281
I20260812 02:39:34.390031 24298 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.188:42281 every 8 connection(s)
I20260812 02:39:34.391443 24219 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502156971023-7698-0/minicluster-data/master-2/data/info.pb
I20260812 02:39:34.395078 24299 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:39:34.399515 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 24219
I20260812 02:39:34.399725 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786502156971023-7698-0/minicluster-data/master-2/wal/instance
I20260812 02:39:34.400246 7698 external_mini_cluster.cc:1428] Setting key 0e2db6ca60dc929f39e0f3afb0ad8d04
I20260812 02:39:34.401963 24299 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.132.190" port: 44813 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 02:39:34 dist-test-slave-mn4r krb5kdc[24030](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502374, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.188@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
Aug 12 02:39:34 dist-test-slave-mn4r krb5kdc[24030](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502372, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.132.188@KRBTEST.COM
I20260812 02:39:34.424988 24303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:34.404626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45498 (local address 127.7.132.190:44813)
0812 02:39:34.405126 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0812 02:39:34.405134 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:39:34.405623 (+ 489us) server_negotiation.cc:407] Connection header received
0812 02:39:34.406029 (+ 406us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:39:34.406034 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:39:34.406166 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:39:34.406277 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:39:34.407516 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:34.408263 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:34.409720 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:34.410195 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:34.410329 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:39:34.417227 (+ 6898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:39:34.417236 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:39:34.417242 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:39:34.417289 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:39:34.421175 (+ 3886us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:39:34.422152 (+ 977us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:39:34.422160 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:39:34.422165 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:39:34.422329 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:39:34.422771 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:39:34.422791 (+ 20us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:39:34.422794 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:39:34.423186 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:39:34.423354 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0812 02:39:34.424247 (+ 893us) server_negotiation.cc:299] Negotiation successful
0812 02:39:34.424511 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":364,"thread_start_us":261,"threads_started":1}
I20260812 02:39:34.426213 24304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:34.404730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:44813 (local address 127.0.0.1:45498)
0812 02:39:34.405478 (+ 748us) negotiation.cc:107] Waiting for socket to connect
0812 02:39:34.405511 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0812 02:39:34.405793 (+ 282us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:39:34.406348 (+ 555us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:39:34.406355 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:39:34.406960 (+ 605us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:39:34.407340 (+ 380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:39:34.407357 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:39:34.408428 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:39:34.408432 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:39:34.409578 (+ 1146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:39:34.409590 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:39:34.410437 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:39:34.410445 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:39:34.410638 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:39:34.414549 (+ 3911us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:39:34.414578 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:39:34.417010 (+ 2432us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:39:34.421327 (+ 4317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:39:34.421339 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:39:34.421358 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:39:34.422009 (+ 651us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:39:34.422461 (+ 452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:39:34.422464 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:39:34.422466 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:39:34.422653 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:39:34.423334 (+ 681us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:39:34.423340 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:39:34.423823 (+ 483us) client_negotiation.cc:769] Sending connection context
0812 02:39:34.424129 (+ 306us) client_negotiation.cc:240] Negotiation successful
0812 02:39:34.424660 (+ 531us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":558,"thread_start_us":417,"threads_started":1}
I20260812 02:39:34.426209 24302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:34.402192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54440 (local address 127.7.132.188:42281)
0812 02:39:34.403079 (+ 887us) server_negotiation.cc:206] Beginning negotiation
0812 02:39:34.403091 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 02:39:34.403134 (+ 43us) server_negotiation.cc:407] Connection header received
0812 02:39:34.405417 (+ 2283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:39:34.405443 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:39:34.405836 (+ 393us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:39:34.406100 (+ 264us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:39:34.407613 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:34.408787 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:34.410114 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:34.410641 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:34.410814 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:39:34.418047 (+ 7233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:39:34.418054 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:39:34.418066 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:39:34.418148 (+ 82us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:39:34.421913 (+ 3765us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:39:34.422851 (+ 938us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:39:34.422863 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:39:34.422865 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:39:34.422983 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:39:34.423436 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:39:34.423441 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:39:34.423444 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:39:34.423911 (+ 467us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:39:34.424206 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0812 02:39:34.424632 (+ 426us) server_negotiation.cc:299] Negotiation successful
0812 02:39:34.424854 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":659,"thread_start_us":414,"threads_started":1}
I20260812 02:39:34.427986 24299 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.132.189" port: 46569 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 02:39:34 dist-test-slave-mn4r krb5kdc[24030](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502374, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.188@KRBTEST.COM for kudu/127.7.132.189@KRBTEST.COM
Aug 12 02:39:34 dist-test-slave-mn4r krb5kdc[24030](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502372, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 02:39:34.443756 24216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:34.428816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45622 (local address 127.7.132.189:46569)
0812 02:39:34.429012 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0812 02:39:34.429026 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 02:39:34.429200 (+ 174us) server_negotiation.cc:407] Connection header received
0812 02:39:34.429308 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:39:34.429314 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:39:34.429389 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:39:34.429504 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:39:34.430330 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:34.431489 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:34.433290 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:34.433965 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:34.434151 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:39:34.437747 (+ 3596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:39:34.437755 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:39:34.437763 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:39:34.437812 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:39:34.440846 (+ 3034us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:39:34.441746 (+ 900us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:39:34.441750 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:39:34.441752 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:39:34.441904 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:39:34.442479 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:39:34.442482 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:39:34.442484 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:39:34.442745 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:39:34.442871 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0812 02:39:34.443308 (+ 437us) server_negotiation.cc:299] Negotiation successful
0812 02:39:34.443471 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 02:39:34.443897 24304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:34.428641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.189:46569 (local address 127.0.0.1:45622)
0812 02:39:34.429114 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0812 02:39:34.429128 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 02:39:34.429216 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:39:34.429537 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:39:34.429541 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:39:34.429924 (+ 383us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:39:34.430169 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:39:34.430183 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:39:34.431651 (+ 1468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:39:34.431655 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:39:34.432982 (+ 1327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:39:34.433000 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:39:34.434228 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:39:34.434232 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:39:34.434420 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:39:34.435100 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:39:34.435118 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:39:34.437525 (+ 2407us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:39:34.441014 (+ 3489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:39:34.441044 (+ 30us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:39:34.441046 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:39:34.441611 (+ 565us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:39:34.442072 (+ 461us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:39:34.442132 (+ 60us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:39:34.442136 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:39:34.442354 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:39:34.442873 (+ 519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:39:34.442878 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:39:34.443059 (+ 181us) client_negotiation.cc:769] Sending connection context
0812 02:39:34.443214 (+ 155us) client_negotiation.cc:240] Negotiation successful
0812 02:39:34.443522 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":382}
I20260812 02:39:34.444878 24299 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.132.188" port: 42281 } has no permanent_uuid. Determining permanent_uuid...
I20260812 02:39:34.446822 24303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:34.429940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45500 (local address 127.7.132.190:44813)
0812 02:39:34.430213 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0812 02:39:34.430221 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:39:34.430240 (+ 19us) server_negotiation.cc:407] Connection header received
0812 02:39:34.430291 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:39:34.430297 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:39:34.430382 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:39:34.430453 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:39:34.431990 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:34.433035 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:34.434782 (+ 1747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:34.435354 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:34.435461 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:39:34.440347 (+ 4886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:39:34.440352 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:39:34.440355 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:39:34.440389 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:39:34.443806 (+ 3417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:39:34.444623 (+ 817us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:39:34.444630 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:39:34.444633 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:39:34.444794 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:39:34.445236 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:39:34.445241 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:39:34.445244 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:39:34.445575 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:39:34.445781 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0812 02:39:34.446237 (+ 456us) server_negotiation.cc:299] Negotiation successful
0812 02:39:34.446488 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111}
Aug 12 02:39:34 dist-test-slave-mn4r krb5kdc[24030](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502374, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.188@KRBTEST.COM for kudu/127.7.132.188@KRBTEST.COM
Aug 12 02:39:34 dist-test-slave-mn4r krb5kdc[24030](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502372, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.132.189@KRBTEST.COM
I20260812 02:39:34.458778 24302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:34.445987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54456 (local address 127.7.132.188:42281)
0812 02:39:34.446175 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0812 02:39:34.446181 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:39:34.446196 (+ 15us) server_negotiation.cc:407] Connection header received
0812 02:39:34.446244 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:39:34.446249 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:39:34.446311 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:39:34.446422 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:39:34.447248 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:34.448064 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:34.449579 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:34.450025 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:34.450209 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:39:34.453184 (+ 2975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:39:34.453189 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:39:34.453192 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:39:34.453223 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:39:34.455958 (+ 2735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:39:34.456668 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:39:34.456673 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:39:34.456675 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:39:34.456787 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:39:34.457185 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:39:34.457192 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:39:34.457199 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:39:34.457504 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:39:34.457660 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0812 02:39:34.458013 (+ 353us) server_negotiation.cc:299] Negotiation successful
0812 02:39:34.458425 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":220,"server-negotiator.queue_time_us":70}
I20260812 02:39:34.458801 24304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:34.445760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:42281 (local address 127.0.0.1:54456)
0812 02:39:34.445947 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0812 02:39:34.445962 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 02:39:34.446135 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:39:34.446474 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:39:34.446478 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:39:34.446953 (+ 475us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:39:34.447104 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:39:34.447113 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:39:34.448188 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:39:34.448193 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:39:34.449442 (+ 1249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:39:34.449454 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:39:34.450217 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:39:34.450222 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:39:34.450465 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:39:34.451024 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:39:34.451040 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:39:34.452973 (+ 1933us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:39:34.456140 (+ 3167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:39:34.456145 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:39:34.456148 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:39:34.456556 (+ 408us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:39:34.456920 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:39:34.456925 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:39:34.456929 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:39:34.457067 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:39:34.457623 (+ 556us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:39:34.457628 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:39:34.457776 (+ 148us) client_negotiation.cc:769] Sending connection context
0812 02:39:34.457925 (+ 149us) client_negotiation.cc:240] Negotiation successful
0812 02:39:34.458289 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":73,"mutex_wait_us":3}
I20260812 02:39:34.463445 24216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:34.448703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45634 (local address 127.7.132.189:46569)
0812 02:39:34.448872 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0812 02:39:34.448878 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:39:34.448896 (+ 18us) server_negotiation.cc:407] Connection header received
0812 02:39:34.449008 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:39:34.449014 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:39:34.449096 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:39:34.449193 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:39:34.450462 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:34.451179 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:34.452452 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:34.452923 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:34.453058 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:39:34.457653 (+ 4595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:39:34.457658 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:39:34.457661 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:39:34.457692 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:39:34.460790 (+ 3098us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:39:34.461437 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:39:34.461449 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:39:34.461452 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:39:34.461585 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:39:34.461999 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:39:34.462003 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:39:34.462005 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:39:34.462322 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:39:34.462622 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0812 02:39:34.462917 (+ 295us) server_negotiation.cc:299] Negotiation successful
0812 02:39:34.463155 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 02:39:34.464493 24299 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3988c7c9497e439eb4f1e35439519f45: Bootstrap starting.
I20260812 02:39:34.468429 7698 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 02:39:34.468472 24299 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3988c7c9497e439eb4f1e35439519f45: Neither blocks nor log segments found. Creating new log.
I20260812 02:39:34.469646 24299 log.cc:824] T 00000000000000000000000000000000 P 3988c7c9497e439eb4f1e35439519f45: Log is configured to *not* fsync() on all Append() calls
I20260812 02:39:34.472676 24299 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3988c7c9497e439eb4f1e35439519f45: No bootstrap required, opened a new log
I20260812 02:39:34.479528 24125 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.132.188" port: 42281 } attempt: 4
I20260812 02:39:34.488744 24299 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3988c7c9497e439eb4f1e35439519f45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6f0690f6e2c43aa849459cd688f58f5" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 44813 } } peers { permanent_uuid: "1389b9e0e56541e1990cf226310c2a87" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 46569 } } peers { permanent_uuid: "3988c7c9497e439eb4f1e35439519f45" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 42281 } }
I20260812 02:39:34.490912 24299 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3988c7c9497e439eb4f1e35439519f45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:39:34.491135 24299 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3988c7c9497e439eb4f1e35439519f45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3988c7c9497e439eb4f1e35439519f45, State: Initialized, Role: FOLLOWER
I20260812 02:39:34.492058 24299 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3988c7c9497e439eb4f1e35439519f45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6f0690f6e2c43aa849459cd688f58f5" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 44813 } } peers { permanent_uuid: "1389b9e0e56541e1990cf226310c2a87" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 46569 } } peers { permanent_uuid: "3988c7c9497e439eb4f1e35439519f45" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 42281 } }
I20260812 02:39:34.498703 24303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:34.473796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45516 (local address 127.7.132.190:44813)
0812 02:39:34.474047 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0812 02:39:34.474054 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:39:34.474630 (+ 576us) server_negotiation.cc:407] Connection header received
0812 02:39:34.474931 (+ 301us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:39:34.474937 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:39:34.475042 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:39:34.475134 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:39:34.478850 (+ 3716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:34.480104 (+ 1254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:34.482902 (+ 2798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:34.483846 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:34.483965 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:39:34.490287 (+ 6322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:39:34.490298 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:39:34.490306 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:39:34.490404 (+ 98us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:39:34.494998 (+ 4594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:39:34.495940 (+ 942us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:39:34.495946 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:39:34.495950 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:39:34.496157 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:39:34.496672 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:39:34.496678 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:39:34.496681 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:39:34.497070 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:39:34.497316 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0812 02:39:34.497978 (+ 662us) server_negotiation.cc:299] Negotiation successful
0812 02:39:34.498235 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
Aug 12 02:39:34 dist-test-slave-mn4r krb5kdc[24030](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502373, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.190@KRBTEST.COM for kudu/127.7.132.188@KRBTEST.COM
I20260812 02:39:34.502820 24317 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3988c7c9497e439eb4f1e35439519f45 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6f0690f6e2c43aa849459cd688f58f5" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 44813 } } peers { permanent_uuid: "1389b9e0e56541e1990cf226310c2a87" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 46569 } } peers { permanent_uuid: "3988c7c9497e439eb4f1e35439519f45" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 42281 } } }
I20260812 02:39:34.503353 24317 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3988c7c9497e439eb4f1e35439519f45 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 02:39:34.504495 24299 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3988c7c9497e439eb4f1e35439519f45 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:39:34.514426 24127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:34.480097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:42281 (local address 127.0.0.1:54474)
0812 02:39:34.480463 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0812 02:39:34.480483 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 02:39:34.480581 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:39:34.493886 (+ 13305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:39:34.493892 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:39:34.494489 (+ 597us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:39:34.494672 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:39:34.494682 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:39:34.497082 (+ 2400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:39:34.497087 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:39:34.498556 (+ 1469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:39:34.498572 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:39:34.499672 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:39:34.499679 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:39:34.499877 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:39:34.500485 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:39:34.500509 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:39:34.503561 (+ 3052us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:39:34.508404 (+ 4843us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:39:34.508410 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:39:34.508415 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:39:34.509043 (+ 628us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:39:34.509610 (+ 567us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:39:34.509617 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:39:34.509623 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:39:34.509804 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:39:34.513033 (+ 3229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:39:34.513042 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:39:34.513274 (+ 232us) client_negotiation.cc:769] Sending connection context
0812 02:39:34.513452 (+ 178us) client_negotiation.cc:240] Negotiation successful
0812 02:39:34.513857 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236}
I20260812 02:39:34.514586 24216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:34.475437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45650 (local address 127.7.132.189:46569)
0812 02:39:34.475675 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0812 02:39:34.475683 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:39:34.486960 (+ 11277us) server_negotiation.cc:407] Connection header received
0812 02:39:34.487241 (+ 281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:39:34.487248 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:39:34.487333 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:39:34.487404 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:39:34.489576 (+ 2172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:34.490810 (+ 1234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:34.502287 (+ 11477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:34.502949 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:34.503084 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:39:34.507088 (+ 4004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:39:34.507094 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:39:34.507097 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:39:34.507138 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:39:34.511069 (+ 3931us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:39:34.511766 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:39:34.511774 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:39:34.511782 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:39:34.511913 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:39:34.512326 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:39:34.512330 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:39:34.512332 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:39:34.512640 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:39:34.512902 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0812 02:39:34.513249 (+ 347us) server_negotiation.cc:299] Negotiation successful
0812 02:39:34.513469 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260812 02:39:34.516010 24315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:34.480793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54474 (local address 127.7.132.188:42281)
0812 02:39:34.493409 (+ 12616us) server_negotiation.cc:206] Beginning negotiation
0812 02:39:34.493416 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:39:34.493441 (+ 25us) server_negotiation.cc:407] Connection header received
0812 02:39:34.493496 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:39:34.493503 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:39:34.493591 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:39:34.495163 (+ 1572us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:39:34.495533 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:34.496914 (+ 1381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:34.498755 (+ 1841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:34.499495 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:34.500179 (+ 684us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:39:34.503785 (+ 3606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:39:34.503790 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:39:34.503794 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:39:34.503859 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:39:34.508208 (+ 4349us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:39:34.509240 (+ 1032us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:39:34.509248 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:39:34.509253 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:39:34.509445 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:39:34.512333 (+ 2888us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:39:34.512342 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:39:34.512348 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:39:34.512838 (+ 490us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:39:34.514914 (+ 2076us) server_negotiation.cc:1035] Waiting for connection context
0812 02:39:34.515083 (+ 169us) server_negotiation.cc:299] Negotiation successful
0812 02:39:34.515383 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":59,"server-negotiator.queue_time_us":12461,"thread_start_us":12896,"threads_started":1}
I20260812 02:39:34.520448 24302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:34.474165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54472 (local address 127.7.132.188:42281)
0812 02:39:34.474342 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0812 02:39:34.474350 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:39:34.485829 (+ 11479us) server_negotiation.cc:407] Connection header received
0812 02:39:34.486042 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:39:34.486052 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:39:34.486177 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:39:34.486309 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:39:34.490980 (+ 4671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:34.492081 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:34.494689 (+ 2608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:34.495889 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:34.496052 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:39:34.505131 (+ 9079us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:39:34.505140 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:39:34.505150 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:39:34.505186 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:39:34.512144 (+ 6958us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:39:34.517539 (+ 5395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:39:34.517545 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:39:34.517550 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:39:34.517731 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:39:34.518240 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:39:34.518246 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:39:34.518249 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:39:34.518675 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:39:34.518970 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0812 02:39:34.519657 (+ 687us) server_negotiation.cc:299] Negotiation successful
0812 02:39:34.519944 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":34}
I20260812 02:39:34.521615 24125 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6f0690f6e2c43aa849459cd688f58f5: Bootstrap starting.
I20260812 02:39:34.525309 24125 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e6f0690f6e2c43aa849459cd688f58f5: Neither blocks nor log segments found. Creating new log.
I20260812 02:39:34.527119 24125 log.cc:824] T 00000000000000000000000000000000 P e6f0690f6e2c43aa849459cd688f58f5: Log is configured to *not* fsync() on all Append() calls
I20260812 02:39:34.530412 24125 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6f0690f6e2c43aa849459cd688f58f5: No bootstrap required, opened a new log
W20260812 02:39:34.532451 24332 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3988c7c9497e439eb4f1e35439519f45: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 02:39:34.532665 24332 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 02:39:34.539664 24125 raft_consensus.cc:348] T 00000000000000000000000000000000 P e6f0690f6e2c43aa849459cd688f58f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6f0690f6e2c43aa849459cd688f58f5" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 44813 } } peers { permanent_uuid: "1389b9e0e56541e1990cf226310c2a87" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 46569 } } peers { permanent_uuid: "3988c7c9497e439eb4f1e35439519f45" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 42281 } }
I20260812 02:39:34.540153 24125 raft_consensus.cc:374] T 00000000000000000000000000000000 P e6f0690f6e2c43aa849459cd688f58f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:39:34.540343 24125 raft_consensus.cc:729] T 00000000000000000000000000000000 P e6f0690f6e2c43aa849459cd688f58f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6f0690f6e2c43aa849459cd688f58f5, State: Initialized, Role: FOLLOWER
I20260812 02:39:34.529389 24219 master_runner.cc:186] Error getting master registration for 127.7.132.190:44813: OK, instance_id { permanent_uuid: "e6f0690f6e2c43aa849459cd688f58f5" instance_seqno: 1786502373323034 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 02:39:34.541069 24219 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 02:39:34.541105 24125 consensus_queue.cc:260] T 00000000000000000000000000000000 P e6f0690f6e2c43aa849459cd688f58f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6f0690f6e2c43aa849459cd688f58f5" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 44813 } } peers { permanent_uuid: "1389b9e0e56541e1990cf226310c2a87" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 46569 } } peers { permanent_uuid: "3988c7c9497e439eb4f1e35439519f45" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 42281 } }
I20260812 02:39:34.542147 24335 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6f0690f6e2c43aa849459cd688f58f5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6f0690f6e2c43aa849459cd688f58f5" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 44813 } } peers { permanent_uuid: "1389b9e0e56541e1990cf226310c2a87" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 46569 } } peers { permanent_uuid: "3988c7c9497e439eb4f1e35439519f45" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 42281 } } }
I20260812 02:39:34.542608 24335 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6f0690f6e2c43aa849459cd688f58f5 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 02:39:34.543267 24125 sys_catalog.cc:564] T 00000000000000000000000000000000 P e6f0690f6e2c43aa849459cd688f58f5 [sys.catalog]: configured and running, proceeding with master startup.
W20260812 02:39:34.553058 24350 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e6f0690f6e2c43aa849459cd688f58f5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 02:39:34.553207 24350 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 02:39:34.552927 24046 master_runner.cc:186] Error getting master registration for 127.7.132.189:46569: OK, instance_id { permanent_uuid: "1389b9e0e56541e1990cf226310c2a87" instance_seqno: 1786502373857004 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 02:39:34.561666 24046 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260812 02:39:34.562440 24071 tablet.cc:2369] T 00000000000000000000000000000000 P e6f0690f6e2c43aa849459cd688f58f5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 02:39:34.592195 24211 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.132.188" port: 42281 } attempt: 5
Aug 12 02:39:34 dist-test-slave-mn4r krb5kdc[24030](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502373, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.189@KRBTEST.COM for kudu/127.7.132.188@KRBTEST.COM
I20260812 02:39:34.607851 24302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:34.592916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54484 (local address 127.7.132.188:42281)
0812 02:39:34.593107 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0812 02:39:34.593113 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:39:34.593132 (+ 19us) server_negotiation.cc:407] Connection header received
0812 02:39:34.593306 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:39:34.593311 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:39:34.593396 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:39:34.593497 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:39:34.594331 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:34.595137 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:34.596382 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:34.596844 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:34.596982 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:39:34.600219 (+ 3237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:39:34.600225 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:39:34.600231 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:39:34.600260 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:39:34.603232 (+ 2972us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:39:34.603911 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:39:34.603918 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:39:34.603921 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:39:34.604120 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:39:34.606373 (+ 2253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:39:34.606381 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:39:34.606384 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:39:34.606695 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:39:34.606866 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0812 02:39:34.607239 (+ 373us) server_negotiation.cc:299] Negotiation successful
0812 02:39:34.607476 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260812 02:39:34.607863 24213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:34.592819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:42281 (local address 127.0.0.1:54484)
0812 02:39:34.593034 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0812 02:39:34.593055 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 02:39:34.593168 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:39:34.593534 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:39:34.593538 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:39:34.593966 (+ 428us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:39:34.594194 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:39:34.594204 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:39:34.595271 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:39:34.595276 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:39:34.596248 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:39:34.596260 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:39:34.597001 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:39:34.597008 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:39:34.597230 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:39:34.597851 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:39:34.597870 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:39:34.600029 (+ 2159us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:39:34.603380 (+ 3351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:39:34.603386 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:39:34.603391 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:39:34.603790 (+ 399us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:39:34.604267 (+ 477us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:39:34.604272 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:39:34.604275 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:39:34.604443 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:39:34.606828 (+ 2385us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:39:34.606833 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:39:34.606992 (+ 159us) client_negotiation.cc:769] Sending connection context
0812 02:39:34.607133 (+ 141us) client_negotiation.cc:240] Negotiation successful
0812 02:39:34.607439 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":72}
I20260812 02:39:34.612686 24211 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1389b9e0e56541e1990cf226310c2a87: Bootstrap starting.
I20260812 02:39:34.616133 24211 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1389b9e0e56541e1990cf226310c2a87: Neither blocks nor log segments found. Creating new log.
I20260812 02:39:34.617447 24211 log.cc:824] T 00000000000000000000000000000000 P 1389b9e0e56541e1990cf226310c2a87: Log is configured to *not* fsync() on all Append() calls
I20260812 02:39:34.621481 24211 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1389b9e0e56541e1990cf226310c2a87: No bootstrap required, opened a new log
W20260812 02:39:34.623771 24244 tablet.cc:2369] T 00000000000000000000000000000000 P 3988c7c9497e439eb4f1e35439519f45: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 02:39:34.629630 24211 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1389b9e0e56541e1990cf226310c2a87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6f0690f6e2c43aa849459cd688f58f5" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 44813 } } peers { permanent_uuid: "1389b9e0e56541e1990cf226310c2a87" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 46569 } } peers { permanent_uuid: "3988c7c9497e439eb4f1e35439519f45" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 42281 } }
I20260812 02:39:34.630002 24211 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1389b9e0e56541e1990cf226310c2a87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:39:34.630153 24211 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1389b9e0e56541e1990cf226310c2a87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1389b9e0e56541e1990cf226310c2a87, State: Initialized, Role: FOLLOWER
I20260812 02:39:34.630831 24211 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1389b9e0e56541e1990cf226310c2a87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6f0690f6e2c43aa849459cd688f58f5" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 44813 } } peers { permanent_uuid: "1389b9e0e56541e1990cf226310c2a87" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 46569 } } peers { permanent_uuid: "3988c7c9497e439eb4f1e35439519f45" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 42281 } }
I20260812 02:39:34.631819 24353 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1389b9e0e56541e1990cf226310c2a87 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6f0690f6e2c43aa849459cd688f58f5" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 44813 } } peers { permanent_uuid: "1389b9e0e56541e1990cf226310c2a87" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 46569 } } peers { permanent_uuid: "3988c7c9497e439eb4f1e35439519f45" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 42281 } } }
I20260812 02:39:34.632306 24353 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1389b9e0e56541e1990cf226310c2a87 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 02:39:34.633119 24211 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1389b9e0e56541e1990cf226310c2a87 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:39:34.654233 24132 master_runner.cc:263] No leader master found from master 1389b9e0e56541e1990cf226310c2a87
I20260812 02:39:34.654394 24132 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260812 02:39:34.660343 24365 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1389b9e0e56541e1990cf226310c2a87: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 02:39:34.660614 24365 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 02:39:34.766366 24317 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3988c7c9497e439eb4f1e35439519f45 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 02:39:34.766716 24317 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3988c7c9497e439eb4f1e35439519f45 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6f0690f6e2c43aa849459cd688f58f5" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 44813 } } peers { permanent_uuid: "1389b9e0e56541e1990cf226310c2a87" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 46569 } } peers { permanent_uuid: "3988c7c9497e439eb4f1e35439519f45" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 42281 } }
I20260812 02:39:34.768390 24317 leader_election.cc:288] T 00000000000000000000000000000000 P 3988c7c9497e439eb4f1e35439519f45 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e6f0690f6e2c43aa849459cd688f58f5 (127.7.132.190:44813), 1389b9e0e56541e1990cf226310c2a87 (127.7.132.189:46569)
I20260812 02:39:34.769774 24182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3988c7c9497e439eb4f1e35439519f45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1389b9e0e56541e1990cf226310c2a87" is_pre_election: true
I20260812 02:39:34.770504 24182 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1389b9e0e56541e1990cf226310c2a87 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3988c7c9497e439eb4f1e35439519f45 in term 0.
I20260812 02:39:34.771371 24240 leader_election.cc:302] T 00000000000000000000000000000000 P 3988c7c9497e439eb4f1e35439519f45 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1389b9e0e56541e1990cf226310c2a87, 3988c7c9497e439eb4f1e35439519f45; no voters:
I20260812 02:39:34.772578 24096 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3988c7c9497e439eb4f1e35439519f45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6f0690f6e2c43aa849459cd688f58f5" is_pre_election: true
I20260812 02:39:34.773229 24096 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e6f0690f6e2c43aa849459cd688f58f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3988c7c9497e439eb4f1e35439519f45 in term 0.
I20260812 02:39:34.774186 24317 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3988c7c9497e439eb4f1e35439519f45 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 02:39:34.774345 24317 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3988c7c9497e439eb4f1e35439519f45 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 02:39:34.774446 24317 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3988c7c9497e439eb4f1e35439519f45 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:39:34.777842 24317 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3988c7c9497e439eb4f1e35439519f45 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6f0690f6e2c43aa849459cd688f58f5" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 44813 } } peers { permanent_uuid: "1389b9e0e56541e1990cf226310c2a87" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 46569 } } peers { permanent_uuid: "3988c7c9497e439eb4f1e35439519f45" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 42281 } }
I20260812 02:39:34.782148 24096 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3988c7c9497e439eb4f1e35439519f45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6f0690f6e2c43aa849459cd688f58f5"
I20260812 02:39:34.782402 24096 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e6f0690f6e2c43aa849459cd688f58f5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:39:34.786264 24096 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e6f0690f6e2c43aa849459cd688f58f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3988c7c9497e439eb4f1e35439519f45 in term 1.
I20260812 02:39:34.786912 24240 leader_election.cc:302] T 00000000000000000000000000000000 P 3988c7c9497e439eb4f1e35439519f45 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3988c7c9497e439eb4f1e35439519f45, e6f0690f6e2c43aa849459cd688f58f5; no voters:
I20260812 02:39:34.790143 24317 leader_election.cc:288] T 00000000000000000000000000000000 P 3988c7c9497e439eb4f1e35439519f45 [CANDIDATE]: Term 1 election: Requested vote from peers e6f0690f6e2c43aa849459cd688f58f5 (127.7.132.190:44813), 1389b9e0e56541e1990cf226310c2a87 (127.7.132.189:46569)
I20260812 02:39:34.790763 24317 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3988c7c9497e439eb4f1e35439519f45 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:39:34.791087 24317 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3988c7c9497e439eb4f1e35439519f45 [term 1 LEADER]: Becoming Leader. State: Replica: 3988c7c9497e439eb4f1e35439519f45, State: Running, Role: LEADER
I20260812 02:39:34.791922 24317 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3988c7c9497e439eb4f1e35439519f45 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6f0690f6e2c43aa849459cd688f58f5" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 44813 } } peers { permanent_uuid: "1389b9e0e56541e1990cf226310c2a87" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 46569 } } peers { permanent_uuid: "3988c7c9497e439eb4f1e35439519f45" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 42281 } }
I20260812 02:39:34.807804 24317 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3988c7c9497e439eb4f1e35439519f45 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3988c7c9497e439eb4f1e35439519f45. Latest consensus state: current_term: 1 leader_uuid: "3988c7c9497e439eb4f1e35439519f45" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6f0690f6e2c43aa849459cd688f58f5" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 44813 } } peers { permanent_uuid: "1389b9e0e56541e1990cf226310c2a87" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 46569 } } peers { permanent_uuid: "3988c7c9497e439eb4f1e35439519f45" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 42281 } } }
I20260812 02:39:34.808727 24317 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3988c7c9497e439eb4f1e35439519f45 [sys.catalog]: This master's current role is: LEADER
I20260812 02:39:34.814332 24182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3988c7c9497e439eb4f1e35439519f45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1389b9e0e56541e1990cf226310c2a87"
I20260812 02:39:34.814594 24182 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1389b9e0e56541e1990cf226310c2a87 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:39:34.818747 24182 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1389b9e0e56541e1990cf226310c2a87 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3988c7c9497e439eb4f1e35439519f45 in term 1.
I20260812 02:39:34.820111 24371 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:39:34.823084 24371 catalog_manager.cc:1484] Initializing Kudu cluster ID...
W20260812 02:39:34.846458 24157 tablet.cc:2369] T 00000000000000000000000000000000 P 1389b9e0e56541e1990cf226310c2a87: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 02:39:34.847366 24182 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1389b9e0e56541e1990cf226310c2a87 [term 1 FOLLOWER]: Refusing update from remote peer 3988c7c9497e439eb4f1e35439519f45: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:39:34.848461 24317 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3988c7c9497e439eb4f1e35439519f45 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1389b9e0e56541e1990cf226310c2a87" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 46569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:39:34.856546 24096 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e6f0690f6e2c43aa849459cd688f58f5 [term 1 FOLLOWER]: Refusing update from remote peer 3988c7c9497e439eb4f1e35439519f45: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:39:34.857745 24317 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3988c7c9497e439eb4f1e35439519f45 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6f0690f6e2c43aa849459cd688f58f5" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 44813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:39:34.890300 24353 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1389b9e0e56541e1990cf226310c2a87 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3988c7c9497e439eb4f1e35439519f45. Latest consensus state: current_term: 1 leader_uuid: "3988c7c9497e439eb4f1e35439519f45" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6f0690f6e2c43aa849459cd688f58f5" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 44813 } } peers { permanent_uuid: "1389b9e0e56541e1990cf226310c2a87" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 46569 } } peers { permanent_uuid: "3988c7c9497e439eb4f1e35439519f45" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 42281 } } }
I20260812 02:39:34.890815 24353 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1389b9e0e56541e1990cf226310c2a87 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 02:39:34.897264 24335 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6f0690f6e2c43aa849459cd688f58f5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3988c7c9497e439eb4f1e35439519f45. Latest consensus state: current_term: 1 leader_uuid: "3988c7c9497e439eb4f1e35439519f45" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6f0690f6e2c43aa849459cd688f58f5" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 44813 } } peers { permanent_uuid: "1389b9e0e56541e1990cf226310c2a87" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 46569 } } peers { permanent_uuid: "3988c7c9497e439eb4f1e35439519f45" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 42281 } } }
I20260812 02:39:34.897583 24335 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6f0690f6e2c43aa849459cd688f58f5 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 02:39:34.900792 24372 mvcc.cc:205] Tried to move back new op lower bound from 7317513727363538944 to 7317513727159361536. Current Snapshot: MvccSnapshot[applied={T|T < 7317513727363538944}]
I20260812 02:39:34.905176 24375 mvcc.cc:205] Tried to move back new op lower bound from 7317513727363538944 to 7317513727159361536. Current Snapshot: MvccSnapshot[applied={T|T < 7317513727363538944}]
I20260812 02:39:34.913169 24371 catalog_manager.cc:1347] Generated new cluster ID: 4d17b026ade448dcae2b745fec804263
I20260812 02:39:34.912955 24335 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6f0690f6e2c43aa849459cd688f58f5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3988c7c9497e439eb4f1e35439519f45" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6f0690f6e2c43aa849459cd688f58f5" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 44813 } } peers { permanent_uuid: "1389b9e0e56541e1990cf226310c2a87" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 46569 } } peers { permanent_uuid: "3988c7c9497e439eb4f1e35439519f45" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 42281 } } }
I20260812 02:39:34.913278 24371 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 02:39:34.913280 24335 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6f0690f6e2c43aa849459cd688f58f5 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 02:39:34.914507 24317 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3988c7c9497e439eb4f1e35439519f45 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3988c7c9497e439eb4f1e35439519f45" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6f0690f6e2c43aa849459cd688f58f5" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 44813 } } peers { permanent_uuid: "1389b9e0e56541e1990cf226310c2a87" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 46569 } } peers { permanent_uuid: "3988c7c9497e439eb4f1e35439519f45" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 42281 } } }
I20260812 02:39:34.914814 24317 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3988c7c9497e439eb4f1e35439519f45 [sys.catalog]: This master's current role is: LEADER
I20260812 02:39:34.933332 24374 mvcc.cc:205] Tried to move back new op lower bound from 7317513727363538944 to 7317513727159361536. Current Snapshot: MvccSnapshot[applied={T|T < 7317513727363538944}]
I20260812 02:39:34.931612 24381 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3988c7c9497e439eb4f1e35439519f45 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3988c7c9497e439eb4f1e35439519f45" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6f0690f6e2c43aa849459cd688f58f5" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 44813 } } peers { permanent_uuid: "1389b9e0e56541e1990cf226310c2a87" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 46569 } } peers { permanent_uuid: "3988c7c9497e439eb4f1e35439519f45" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 42281 } } }
I20260812 02:39:34.939508 24381 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3988c7c9497e439eb4f1e35439519f45 [sys.catalog]: This master's current role is: LEADER
I20260812 02:39:34.943064 24353 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1389b9e0e56541e1990cf226310c2a87 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3988c7c9497e439eb4f1e35439519f45" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6f0690f6e2c43aa849459cd688f58f5" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 44813 } } peers { permanent_uuid: "1389b9e0e56541e1990cf226310c2a87" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 46569 } } peers { permanent_uuid: "3988c7c9497e439eb4f1e35439519f45" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 42281 } } }
I20260812 02:39:34.943331 24353 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1389b9e0e56541e1990cf226310c2a87 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 02:39:34.954310 24371 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 02:39:34.956228 24371 catalog_manager.cc:1504] Loading token signing keys...
I20260812 02:39:34.981096 24371 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3988c7c9497e439eb4f1e35439519f45: Generated new TSK 0
I20260812 02:39:34.982322 24371 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 02:39:35.555331 24350 catalog_manager.cc:1259] Loaded cluster ID: 4d17b026ade448dcae2b745fec804263
I20260812 02:39:35.555437 24350 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e6f0690f6e2c43aa849459cd688f58f5: loading cluster ID for follower catalog manager: success
I20260812 02:39:35.557803 24350 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e6f0690f6e2c43aa849459cd688f58f5: acquiring CA information for follower catalog manager: success
I20260812 02:39:35.559639 24350 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e6f0690f6e2c43aa849459cd688f58f5: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 02:39:35.662722 24365 catalog_manager.cc:1259] Loaded cluster ID: 4d17b026ade448dcae2b745fec804263
I20260812 02:39:35.662864 24365 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1389b9e0e56541e1990cf226310c2a87: loading cluster ID for follower catalog manager: success
I20260812 02:39:35.665318 24365 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1389b9e0e56541e1990cf226310c2a87: acquiring CA information for follower catalog manager: success
I20260812 02:39:35.667140 24365 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1389b9e0e56541e1990cf226310c2a87: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 02:39:36.125892 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 24046
I20260812 02:39:36.128032 24119 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:39:36.241250 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 24046
I20260812 02:39:36.268579 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 24132
I20260812 02:39:36.269737 24205 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:39:36.378770 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 24132
I20260812 02:39:36.405586 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 24219
I20260812 02:39:36.406950 24294 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:39:36.517266 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 24219
2026-08-12T02:39:36Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (5145 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786502156971023-7698-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 02:39:36 dist-test-slave-mn4r krb5kdc[24396](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 02:39:36 dist-test-slave-mn4r krb5kdc[24396](info): set up 2 sockets
Aug 12 02:39:36 dist-test-slave-mn4r krb5kdc[24396](info): commencing operation
krb5kdc: starting...
W20260812 02:39:37.734697 7698 mini_kdc.cc:121] Time spent starting KDC: real 1.170s user 0.001s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786502156971023-7698-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786502156971023-7698-0/krb5kdc/test-user.keytab.
Aug 12 02:39:37 dist-test-slave-mn4r krb5kdc[24396](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502377, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T02:39:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T02:39:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.132.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
I20260812 02:39:38.023394 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786502156971023-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786502156971023-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786502156971023-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:34903
--webserver_interface=127.7.132.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:38501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.132.190:34903
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:39:38.367008 24411 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:39:38.367368 24411 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:39:38.367537 24411 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:39:38.375576 24411 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:39:38.375725 24411 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:39:38.375813 24411 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:39:38.375919 24411 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:39:38.376014 24411 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:39:38.386940 24411 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:38501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786502156971023-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786502156971023-7698-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.132.190:34903
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:34903
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.132.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786502156971023-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:39:38.388383 24411 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:39:38.390430 24411 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:39:38.403528 24420 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:39:38.403477 24418 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:39:38.403568 24423 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:39:38.404192 24411 server_base.cc:1034] running on GCE node
I20260812 02:39:38.405035 24411 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:39:38.406572 24411 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:39:38.407840 24411 hybrid_clock.cc:648] HybridClock initialized: now 1786502378407791 us; error 47 us; skew 500 ppm
I20260812 02:39:38.408248 24411 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:39:38 dist-test-slave-mn4r krb5kdc[24396](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502378, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:39:38.411453 24411 init.cc:377] Logged in from keytab as kudu/127.7.132.190@KRBTEST.COM (short username kudu)
I20260812 02:39:38.413033 24411 webserver.cc:460] Webserver started at http://127.7.132.190:35775/ using document root <none> and password file <none>
I20260812 02:39:38.413743 24411 fs_manager.cc:359] Metadata directory not provided
I20260812 02:39:38.413900 24411 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:39:38.414287 24411 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:39:38.417348 24411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786502156971023-7698-0/minicluster-data/master-0/data/instance:
uuid: "fb18fdd4601e46d680b7b767db23ea67"
format_stamp: "Formatted at 2026-08-12 02:39:38 on dist-test-slave-mn4r"
server_key: "87e5a7fa4a070ce6ece47a2b8e1f12fd"
server_key_iv: "9fed2fb46ad06d563eaf862fedb19f45"
server_key_version: "encryptionkey@0"
I20260812 02:39:38.418267 24411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786502156971023-7698-0/minicluster-data/master-0/wal/instance:
uuid: "fb18fdd4601e46d680b7b767db23ea67"
format_stamp: "Formatted at 2026-08-12 02:39:38 on dist-test-slave-mn4r"
server_key: "87e5a7fa4a070ce6ece47a2b8e1f12fd"
server_key_iv: "9fed2fb46ad06d563eaf862fedb19f45"
server_key_version: "encryptionkey@0"
I20260812 02:39:38.423655 24411 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260812 02:39:38.427269 24432 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:39:38.428637 24411 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 02:39:38.428838 24411 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786502156971023-7698-0/minicluster-data/master-0/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786502156971023-7698-0/minicluster-data/master-0/wal
uuid: "fb18fdd4601e46d680b7b767db23ea67"
format_stamp: "Formatted at 2026-08-12 02:39:38 on dist-test-slave-mn4r"
server_key: "87e5a7fa4a070ce6ece47a2b8e1f12fd"
server_key_iv: "9fed2fb46ad06d563eaf862fedb19f45"
server_key_version: "encryptionkey@0"
I20260812 02:39:38.429060 24411 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786502156971023-7698-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786502156971023-7698-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786502156971023-7698-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:39:38.450708 24411 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:39:38.454958 24411 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:39:38.455265 24411 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:39:38.474910 24411 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.190:34903
I20260812 02:39:38.474931 24493 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.190:34903 every 8 connection(s)
I20260812 02:39:38.476377 24411 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
I20260812 02:39:38.479677 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 24411
I20260812 02:39:38.479934 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786502156971023-7698-0/minicluster-data/master-0/wal/instance
I20260812 02:39:38.480666 24494 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:39:38.480827 7698 external_mini_cluster.cc:1428] Setting key adcf8dd0602d26ccc6ce5001a43538d7
I20260812 02:39:38.495919 24494 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb18fdd4601e46d680b7b767db23ea67: Bootstrap starting.
Aug 12 02:39:38 dist-test-slave-mn4r krb5kdc[24396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502377, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 02:39:38.499318 24494 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fb18fdd4601e46d680b7b767db23ea67: Neither blocks nor log segments found. Creating new log.
I20260812 02:39:38.500469 24494 log.cc:824] T 00000000000000000000000000000000 P fb18fdd4601e46d680b7b767db23ea67: Log is configured to *not* fsync() on all Append() calls
I20260812 02:39:38.503639 24494 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb18fdd4601e46d680b7b767db23ea67: No bootstrap required, opened a new log
I20260812 02:39:38.505960 24497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:38.482865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50320 (local address 127.7.132.190:34903)
0812 02:39:38.483760 (+ 895us) server_negotiation.cc:206] Beginning negotiation
0812 02:39:38.483772 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 02:39:38.483818 (+ 46us) server_negotiation.cc:407] Connection header received
0812 02:39:38.485690 (+ 1872us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:39:38.485712 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:39:38.486179 (+ 467us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:39:38.486545 (+ 366us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:39:38.488467 (+ 1922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:38.489809 (+ 1342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:38.491791 (+ 1982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:38.492577 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:38.492733 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:39:38.497990 (+ 5257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:39:38.497999 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:39:38.498014 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:39:38.498053 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:39:38.501544 (+ 3491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:39:38.502475 (+ 931us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:39:38.502483 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:39:38.502490 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:39:38.502662 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:39:38.503101 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:39:38.503105 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:39:38.503107 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:39:38.503643 (+ 536us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:39:38.503971 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0812 02:39:38.504458 (+ 487us) server_negotiation.cc:299] Negotiation successful
0812 02:39:38.504917 (+ 459us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":689,"thread_start_us":393,"threads_started":1}
I20260812 02:39:38.510057 7698 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 02:39:38.512007 24494 raft_consensus.cc:348] T 00000000000000000000000000000000 P fb18fdd4601e46d680b7b767db23ea67 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb18fdd4601e46d680b7b767db23ea67" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 34903 } }
I20260812 02:39:38.512501 24494 raft_consensus.cc:374] T 00000000000000000000000000000000 P fb18fdd4601e46d680b7b767db23ea67 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:39:38.512660 24494 raft_consensus.cc:729] T 00000000000000000000000000000000 P fb18fdd4601e46d680b7b767db23ea67 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb18fdd4601e46d680b7b767db23ea67, State: Initialized, Role: FOLLOWER
I20260812 02:39:38.513269 24494 consensus_queue.cc:260] T 00000000000000000000000000000000 P fb18fdd4601e46d680b7b767db23ea67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb18fdd4601e46d680b7b767db23ea67" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 34903 } }
I20260812 02:39:38.513585 24494 raft_consensus.cc:388] T 00000000000000000000000000000000 P fb18fdd4601e46d680b7b767db23ea67 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 02:39:38.513742 24494 raft_consensus.cc:482] T 00000000000000000000000000000000 P fb18fdd4601e46d680b7b767db23ea67 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 02:39:38.513931 24494 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fb18fdd4601e46d680b7b767db23ea67 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:39:38.516211 24494 raft_consensus.cc:504] T 00000000000000000000000000000000 P fb18fdd4601e46d680b7b767db23ea67 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb18fdd4601e46d680b7b767db23ea67" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 34903 } }
I20260812 02:39:38.516736 24494 leader_election.cc:302] T 00000000000000000000000000000000 P fb18fdd4601e46d680b7b767db23ea67 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fb18fdd4601e46d680b7b767db23ea67; no voters:
I20260812 02:39:38.517388 24494 leader_election.cc:288] T 00000000000000000000000000000000 P fb18fdd4601e46d680b7b767db23ea67 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 02:39:38.517556 24500 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fb18fdd4601e46d680b7b767db23ea67 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:39:38.518271 24500 raft_consensus.cc:686] T 00000000000000000000000000000000 P fb18fdd4601e46d680b7b767db23ea67 [term 1 LEADER]: Becoming Leader. State: Replica: fb18fdd4601e46d680b7b767db23ea67, State: Running, Role: LEADER
I20260812 02:39:38.518808 24500 consensus_queue.cc:237] T 00000000000000000000000000000000 P fb18fdd4601e46d680b7b767db23ea67 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb18fdd4601e46d680b7b767db23ea67" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 34903 } }
I20260812 02:39:38.519562 24494 sys_catalog.cc:564] T 00000000000000000000000000000000 P fb18fdd4601e46d680b7b767db23ea67 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:39:38.521654 24501 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb18fdd4601e46d680b7b767db23ea67 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fb18fdd4601e46d680b7b767db23ea67" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb18fdd4601e46d680b7b767db23ea67" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 34903 } } }
I20260812 02:39:38.522148 24501 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb18fdd4601e46d680b7b767db23ea67 [sys.catalog]: This master's current role is: LEADER
I20260812 02:39:38.523061 24510 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:39:38.522886 24502 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb18fdd4601e46d680b7b767db23ea67 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fb18fdd4601e46d680b7b767db23ea67. Latest consensus state: current_term: 1 leader_uuid: "fb18fdd4601e46d680b7b767db23ea67" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb18fdd4601e46d680b7b767db23ea67" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 34903 } } }
I20260812 02:39:38.523228 24502 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb18fdd4601e46d680b7b767db23ea67 [sys.catalog]: This master's current role is: LEADER
I20260812 02:39:38.528412 24510 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 02:39:38.537722 24510 catalog_manager.cc:1347] Generated new cluster ID: f257197a35c44338b95b701b77e422d4
I20260812 02:39:38.537883 24510 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 02:39:38.549923 24510 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 02:39:38.551213 24510 catalog_manager.cc:1504] Loading token signing keys...
I20260812 02:39:38.556999 24510 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fb18fdd4601e46d680b7b767db23ea67: Generated new TSK 0
I20260812 02:39:38.557895 24510 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 02:39:38.570433 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 24411
I20260812 02:39:38.571642 24488 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:39:38.667459 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 24411
2026-08-12T02:39:38Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (2145 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 02:39:38 dist-test-slave-mn4r krb5kdc[24530](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 02:39:38 dist-test-slave-mn4r krb5kdc[24530](info): set up 2 sockets
Aug 12 02:39:38 dist-test-slave-mn4r krb5kdc[24530](info): commencing operation
krb5kdc: starting...
W20260812 02:39:39.893353 7698 mini_kdc.cc:121] Time spent starting KDC: real 1.184s user 0.000s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/krb5kdc/test-user.keytab.
Aug 12 02:39:40 dist-test-slave-mn4r krb5kdc[24530](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502380, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T02:39:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T02:39:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.132.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
I20260812 02:39:40.185688 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:44921
--webserver_interface=127.7.132.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:42213
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.132.190:44921
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:39:40.537779 24546 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:39:40.538149 24546 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:39:40.538331 24546 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:39:40.545753 24546 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:39:40.545862 24546 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:39:40.546008 24546 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:39:40.546128 24546 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:39:40.546191 24546 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:39:40.556154 24546 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:42213
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.132.190:44921
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:44921
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.132.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:39:40.557560 24546 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:39:40.558964 24546 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:39:40.571566 24552 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:39:40.571647 24551 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:39:40.571620 24555 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:39:40.572610 24546 server_base.cc:1034] running on GCE node
I20260812 02:39:40.573410 24546 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:39:40.574929 24546 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:39:40.576170 24546 hybrid_clock.cc:648] HybridClock initialized: now 1786502380576119 us; error 40 us; skew 500 ppm
I20260812 02:39:40.576536 24546 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:39:40 dist-test-slave-mn4r krb5kdc[24530](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502380, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:39:40.584952 24546 init.cc:377] Logged in from keytab as kudu/127.7.132.190@KRBTEST.COM (short username kudu)
I20260812 02:39:40.586670 24546 webserver.cc:460] Webserver started at http://127.7.132.190:45015/ using document root <none> and password file <none>
I20260812 02:39:40.587414 24546 fs_manager.cc:359] Metadata directory not provided
I20260812 02:39:40.587577 24546 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:39:40.587900 24546 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:39:40.591109 24546 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/master-0/data/instance:
uuid: "ce6e17ddb54e4dc39dc6f8ef9c7f2d94"
format_stamp: "Formatted at 2026-08-12 02:39:40 on dist-test-slave-mn4r"
server_key: "9ba3905bf917b4c9aa9d00b47d3523df"
server_key_iv: "0338562476a92610dbf99c24a56e6b70"
server_key_version: "encryptionkey@0"
I20260812 02:39:40.591881 24546 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/master-0/wal/instance:
uuid: "ce6e17ddb54e4dc39dc6f8ef9c7f2d94"
format_stamp: "Formatted at 2026-08-12 02:39:40 on dist-test-slave-mn4r"
server_key: "9ba3905bf917b4c9aa9d00b47d3523df"
server_key_iv: "0338562476a92610dbf99c24a56e6b70"
server_key_version: "encryptionkey@0"
I20260812 02:39:40.597226 24546 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260812 02:39:40.600835 24565 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:39:40.602411 24546 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 02:39:40.602562 24546 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/master-0/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/master-0/wal
uuid: "ce6e17ddb54e4dc39dc6f8ef9c7f2d94"
format_stamp: "Formatted at 2026-08-12 02:39:40 on dist-test-slave-mn4r"
server_key: "9ba3905bf917b4c9aa9d00b47d3523df"
server_key_iv: "0338562476a92610dbf99c24a56e6b70"
server_key_version: "encryptionkey@0"
I20260812 02:39:40.602748 24546 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:39:40.623805 24546 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:39:40.627800 24546 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:39:40.628054 24546 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:39:40.646492 24546 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.190:44921
I20260812 02:39:40.646539 24627 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.190:44921 every 8 connection(s)
I20260812 02:39:40.647902 24546 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
I20260812 02:39:40.651558 24628 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:39:40.654268 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 24546
I20260812 02:39:40.654507 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/master-0/wal/instance
I20260812 02:39:40.655409 7698 external_mini_cluster.cc:1428] Setting key b189ba71d33d9ee380b72a9e571f09f5
I20260812 02:39:40.663740 24628 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce6e17ddb54e4dc39dc6f8ef9c7f2d94: Bootstrap starting.
I20260812 02:39:40.666775 24628 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ce6e17ddb54e4dc39dc6f8ef9c7f2d94: Neither blocks nor log segments found. Creating new log.
I20260812 02:39:40.667753 24628 log.cc:824] T 00000000000000000000000000000000 P ce6e17ddb54e4dc39dc6f8ef9c7f2d94: Log is configured to *not* fsync() on all Append() calls
Aug 12 02:39:40 dist-test-slave-mn4r krb5kdc[24530](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502380, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 02:39:40.670667 24628 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce6e17ddb54e4dc39dc6f8ef9c7f2d94: No bootstrap required, opened a new log
I20260812 02:39:40.678371 24631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:40.657776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42700 (local address 127.7.132.190:44921)
0812 02:39:40.658620 (+ 844us) server_negotiation.cc:206] Beginning negotiation
0812 02:39:40.658633 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 02:39:40.658673 (+ 40us) server_negotiation.cc:407] Connection header received
0812 02:39:40.660311 (+ 1638us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:39:40.660334 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:39:40.660717 (+ 383us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:39:40.661134 (+ 417us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:39:40.662721 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:40.664072 (+ 1351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:40.665430 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:40.666137 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:40.666353 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:39:40.670867 (+ 4514us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:39:40.670875 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:39:40.670890 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:39:40.670925 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:39:40.674177 (+ 3252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:39:40.674899 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:39:40.674914 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:39:40.674920 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:39:40.675093 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:39:40.675491 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:39:40.675500 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:39:40.675511 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:39:40.676056 (+ 545us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:39:40.676403 (+ 347us) server_negotiation.cc:1035] Waiting for connection context
0812 02:39:40.676851 (+ 448us) server_negotiation.cc:299] Negotiation successful
0812 02:39:40.677291 (+ 440us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":619,"thread_start_us":399,"threads_started":1}
I20260812 02:39:40.678725 24628 raft_consensus.cc:348] T 00000000000000000000000000000000 P ce6e17ddb54e4dc39dc6f8ef9c7f2d94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce6e17ddb54e4dc39dc6f8ef9c7f2d94" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 44921 } }
I20260812 02:39:40.679196 24628 raft_consensus.cc:374] T 00000000000000000000000000000000 P ce6e17ddb54e4dc39dc6f8ef9c7f2d94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:39:40.679311 24628 raft_consensus.cc:729] T 00000000000000000000000000000000 P ce6e17ddb54e4dc39dc6f8ef9c7f2d94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce6e17ddb54e4dc39dc6f8ef9c7f2d94, State: Initialized, Role: FOLLOWER
I20260812 02:39:40.679939 24628 consensus_queue.cc:260] T 00000000000000000000000000000000 P ce6e17ddb54e4dc39dc6f8ef9c7f2d94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce6e17ddb54e4dc39dc6f8ef9c7f2d94" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 44921 } }
I20260812 02:39:40.680212 24628 raft_consensus.cc:388] T 00000000000000000000000000000000 P ce6e17ddb54e4dc39dc6f8ef9c7f2d94 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 02:39:40.680372 24628 raft_consensus.cc:482] T 00000000000000000000000000000000 P ce6e17ddb54e4dc39dc6f8ef9c7f2d94 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 02:39:40.680552 24628 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ce6e17ddb54e4dc39dc6f8ef9c7f2d94 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:39:40.683068 24628 raft_consensus.cc:504] T 00000000000000000000000000000000 P ce6e17ddb54e4dc39dc6f8ef9c7f2d94 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce6e17ddb54e4dc39dc6f8ef9c7f2d94" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 44921 } }
I20260812 02:39:40.683545 24628 leader_election.cc:302] T 00000000000000000000000000000000 P ce6e17ddb54e4dc39dc6f8ef9c7f2d94 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ce6e17ddb54e4dc39dc6f8ef9c7f2d94; no voters:
I20260812 02:39:40.684127 24628 leader_election.cc:288] T 00000000000000000000000000000000 P ce6e17ddb54e4dc39dc6f8ef9c7f2d94 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 02:39:40.684266 24635 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ce6e17ddb54e4dc39dc6f8ef9c7f2d94 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:39:40.685020 24635 raft_consensus.cc:686] T 00000000000000000000000000000000 P ce6e17ddb54e4dc39dc6f8ef9c7f2d94 [term 1 LEADER]: Becoming Leader. State: Replica: ce6e17ddb54e4dc39dc6f8ef9c7f2d94, State: Running, Role: LEADER
I20260812 02:39:40.685598 24635 consensus_queue.cc:237] T 00000000000000000000000000000000 P ce6e17ddb54e4dc39dc6f8ef9c7f2d94 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce6e17ddb54e4dc39dc6f8ef9c7f2d94" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 44921 } }
I20260812 02:39:40.686349 24628 sys_catalog.cc:564] T 00000000000000000000000000000000 P ce6e17ddb54e4dc39dc6f8ef9c7f2d94 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:39:40.688475 24636 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce6e17ddb54e4dc39dc6f8ef9c7f2d94 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ce6e17ddb54e4dc39dc6f8ef9c7f2d94" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce6e17ddb54e4dc39dc6f8ef9c7f2d94" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 44921 } } }
I20260812 02:39:40.689023 24636 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce6e17ddb54e4dc39dc6f8ef9c7f2d94 [sys.catalog]: This master's current role is: LEADER
I20260812 02:39:40.688951 24637 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce6e17ddb54e4dc39dc6f8ef9c7f2d94 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ce6e17ddb54e4dc39dc6f8ef9c7f2d94. Latest consensus state: current_term: 1 leader_uuid: "ce6e17ddb54e4dc39dc6f8ef9c7f2d94" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce6e17ddb54e4dc39dc6f8ef9c7f2d94" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 44921 } } }
I20260812 02:39:40.689323 24637 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce6e17ddb54e4dc39dc6f8ef9c7f2d94 [sys.catalog]: This master's current role is: LEADER
I20260812 02:39:40.691388 24641 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:39:40.696928 24641 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 02:39:40.705870 24641 catalog_manager.cc:1347] Generated new cluster ID: a19eed9dfb2546c89c27bf5be3380eb7
I20260812 02:39:40.706104 24641 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 02:39:40.718241 24641 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 02:39:40.719610 24641 catalog_manager.cc:1504] Loading token signing keys...
I20260812 02:39:40.725785 24641 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ce6e17ddb54e4dc39dc6f8ef9c7f2d94: Generated new TSK 0
I20260812 02:39:40.726719 24641 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.132.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.129.keytab.
Entry for principal kudu/127.7.132.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.129.keytab.
I20260812 02:39:40.803189 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.129:0
--local_ip_for_outbound_sockets=127.7.132.129
--webserver_interface=127.7.132.129
--webserver_port=0
--tserver_master_addrs=127.7.132.190:44921
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.129.keytab
--principal=kudu/127.7.132.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:42213
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:39:41.154455 24658 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:39:41.154737 24658 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:39:41.154906 24658 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:39:41.162729 24658 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:39:41.162832 24658 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:39:41.163008 24658 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.129
I20260812 02:39:41.172977 24658 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:42213
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.129.keytab
--principal=kudu/127.7.132.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.132.129
--webserver_port=0
--tserver_master_addrs=127.7.132.190:44921
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.129
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:39:41.174392 24658 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:39:41.175773 24658 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:39:41.190354 24664 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:39:41.190376 24665 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:39:41.190951 24668 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:39:41.195523 24658 server_base.cc:1034] running on GCE node
I20260812 02:39:41.196130 24658 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:39:41.197068 24658 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:39:41.198292 24658 hybrid_clock.cc:648] HybridClock initialized: now 1786502381198246 us; error 36 us; skew 500 ppm
I20260812 02:39:41.198621 24658 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:39:41 dist-test-slave-mn4r krb5kdc[24530](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502381, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:39:41.201692 24658 init.cc:377] Logged in from keytab as kudu/127.7.132.129@KRBTEST.COM (short username kudu)
I20260812 02:39:41.203276 24658 webserver.cc:460] Webserver started at http://127.7.132.129:34017/ using document root <none> and password file <none>
I20260812 02:39:41.204010 24658 fs_manager.cc:359] Metadata directory not provided
I20260812 02:39:41.204171 24658 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:39:41.204491 24658 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:39:41.207619 24658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-0/data/instance:
uuid: "a78f88301ed54f48bf9e17f58fd62d5e"
format_stamp: "Formatted at 2026-08-12 02:39:41 on dist-test-slave-mn4r"
server_key: "7b5f87b6450b31d2c4b0e385768e0548"
server_key_iv: "90cbe381b2dc9e8fe4cac009c9028cfa"
server_key_version: "encryptionkey@0"
I20260812 02:39:41.208377 24658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-0/wal/instance:
uuid: "a78f88301ed54f48bf9e17f58fd62d5e"
format_stamp: "Formatted at 2026-08-12 02:39:41 on dist-test-slave-mn4r"
server_key: "7b5f87b6450b31d2c4b0e385768e0548"
server_key_iv: "90cbe381b2dc9e8fe4cac009c9028cfa"
server_key_version: "encryptionkey@0"
I20260812 02:39:41.213460 24658 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260812 02:39:41.216952 24678 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:39:41.218442 24658 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 02:39:41.218619 24658 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-0/wal
uuid: "a78f88301ed54f48bf9e17f58fd62d5e"
format_stamp: "Formatted at 2026-08-12 02:39:41 on dist-test-slave-mn4r"
server_key: "7b5f87b6450b31d2c4b0e385768e0548"
server_key_iv: "90cbe381b2dc9e8fe4cac009c9028cfa"
server_key_version: "encryptionkey@0"
I20260812 02:39:41.218814 24658 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:39:41.240155 24658 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:39:41.244131 24658 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:39:41.244431 24658 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:39:41.245404 24658 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:39:41.246798 24658 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:39:41.246908 24658 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:39:41.247068 24658 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:39:41.247143 24658 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:39:41.280556 24658 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.129:41103
I20260812 02:39:41.280606 24805 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.129:41103 every 8 connection(s)
I20260812 02:39:41.281915 24658 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-0/data/info.pb
I20260812 02:39:41.285921 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 24658
I20260812 02:39:41.286180 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-0/wal/instance
I20260812 02:39:41.287070 7698 external_mini_cluster.cc:1428] Setting key 5175ad9c6f211bf8ee9ac9af5ca42f62
Aug 12 02:39:41 dist-test-slave-mn4r krb5kdc[24530](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502381, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.129@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 02:39:41.306524 24809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:41.284562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:52359 (local address 127.7.132.190:44921)
0812 02:39:41.285067 (+ 505us) server_negotiation.cc:206] Beginning negotiation
0812 02:39:41.285072 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:39:41.286159 (+ 1087us) server_negotiation.cc:407] Connection header received
0812 02:39:41.288640 (+ 2481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:39:41.288645 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:39:41.288718 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:39:41.288845 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:39:41.290975 (+ 2130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:41.291833 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:41.294480 (+ 2647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:41.294869 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:41.294971 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:39:41.299406 (+ 4435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:39:41.299416 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:39:41.299423 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:39:41.299473 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:39:41.302729 (+ 3256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:39:41.303760 (+ 1031us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:39:41.303764 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:39:41.303767 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:39:41.303893 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:39:41.304402 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:39:41.304406 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:39:41.304408 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:39:41.304669 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:39:41.304781 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0812 02:39:41.305901 (+ 1120us) server_negotiation.cc:299] Negotiation successful
0812 02:39:41.306225 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":401,"thread_start_us":271,"threads_started":1}
I20260812 02:39:41.307912 24810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:41.284986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:44921 (local address 127.7.132.129:52359)
0812 02:39:41.285881 (+ 895us) negotiation.cc:107] Waiting for socket to connect
0812 02:39:41.285941 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0812 02:39:41.288370 (+ 2429us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:39:41.289124 (+ 754us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:39:41.289138 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:39:41.289889 (+ 751us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:39:41.290781 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:39:41.290817 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:39:41.292066 (+ 1249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:39:41.292074 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:39:41.294313 (+ 2239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:39:41.294331 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:39:41.295114 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:39:41.295124 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:39:41.295466 (+ 342us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:39:41.296269 (+ 803us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:39:41.296306 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:39:41.299135 (+ 2829us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:39:41.302889 (+ 3754us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:39:41.302901 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:39:41.302922 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:39:41.303616 (+ 694us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:39:41.304005 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:39:41.304013 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:39:41.304019 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:39:41.304285 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:39:41.304798 (+ 513us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:39:41.304810 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:39:41.305338 (+ 528us) client_negotiation.cc:769] Sending connection context
0812 02:39:41.305691 (+ 353us) client_negotiation.cc:240] Negotiation successful
0812 02:39:41.306344 (+ 653us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":560,"thread_start_us":314,"threads_started":1}
I20260812 02:39:41.309633 24806 heartbeater.cc:347] Connected to a master server at 127.7.132.190:44921
I20260812 02:39:41.309962 24806 heartbeater.cc:464] Registering TS with master...
I20260812 02:39:41.310762 24806 heartbeater.cc:511] Master 127.7.132.190:44921 requested a full tablet report, sending...
I20260812 02:39:41.312894 24585 ts_manager.cc:194] Registered new tserver with Master: a78f88301ed54f48bf9e17f58fd62d5e (127.7.132.129:41103)
I20260812 02:39:41.314505 24585 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.129@KRBTEST.COM'} at 127.7.132.129:52359
WARNING: no policy specified for kudu/127.7.132.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.130.keytab.
Entry for principal kudu/127.7.132.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.130.keytab.
I20260812 02:39:41.358189 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.130:0
--local_ip_for_outbound_sockets=127.7.132.130
--webserver_interface=127.7.132.130
--webserver_port=0
--tserver_master_addrs=127.7.132.190:44921
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.130.keytab
--principal=kudu/127.7.132.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:42213
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:39:41.703969 24814 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:39:41.704308 24814 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:39:41.704480 24814 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:39:41.711971 24814 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:39:41.712126 24814 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:39:41.712313 24814 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.130
I20260812 02:39:41.721747 24814 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:42213
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.130.keytab
--principal=kudu/127.7.132.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.132.130
--webserver_port=0
--tserver_master_addrs=127.7.132.190:44921
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.130
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:39:41.723191 24814 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:39:41.724524 24814 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:39:41.737808 24823 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:39:41.737864 24825 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:39:41.737902 24822 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:39:41.739013 24814 server_base.cc:1034] running on GCE node
I20260812 02:39:41.739656 24814 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:39:41.740602 24814 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:39:41.741832 24814 hybrid_clock.cc:648] HybridClock initialized: now 1786502381741782 us; error 44 us; skew 500 ppm
I20260812 02:39:41.742230 24814 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:39:41 dist-test-slave-mn4r krb5kdc[24530](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502381, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:39:41.745364 24814 init.cc:377] Logged in from keytab as kudu/127.7.132.130@KRBTEST.COM (short username kudu)
I20260812 02:39:41.746814 24814 webserver.cc:460] Webserver started at http://127.7.132.130:36867/ using document root <none> and password file <none>
I20260812 02:39:41.747531 24814 fs_manager.cc:359] Metadata directory not provided
I20260812 02:39:41.747675 24814 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:39:41.747991 24814 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:39:41.750952 24814 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-1/data/instance:
uuid: "9a848619566249798e7b9f2d4118c664"
format_stamp: "Formatted at 2026-08-12 02:39:41 on dist-test-slave-mn4r"
server_key: "8d3147d5c2e5775b5e5295a7d3f1933e"
server_key_iv: "caf67665118634047b0df87a316228e8"
server_key_version: "encryptionkey@0"
I20260812 02:39:41.751717 24814 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-1/wal/instance:
uuid: "9a848619566249798e7b9f2d4118c664"
format_stamp: "Formatted at 2026-08-12 02:39:41 on dist-test-slave-mn4r"
server_key: "8d3147d5c2e5775b5e5295a7d3f1933e"
server_key_iv: "caf67665118634047b0df87a316228e8"
server_key_version: "encryptionkey@0"
I20260812 02:39:41.756906 24814 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260812 02:39:41.760325 24833 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:39:41.761610 24814 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 02:39:41.761792 24814 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-1/wal
uuid: "9a848619566249798e7b9f2d4118c664"
format_stamp: "Formatted at 2026-08-12 02:39:41 on dist-test-slave-mn4r"
server_key: "8d3147d5c2e5775b5e5295a7d3f1933e"
server_key_iv: "caf67665118634047b0df87a316228e8"
server_key_version: "encryptionkey@0"
I20260812 02:39:41.762050 24814 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:39:41.784368 24814 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:39:41.788892 24814 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:39:41.789204 24814 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:39:41.790318 24814 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:39:41.791857 24814 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:39:41.791963 24814 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:39:41.792126 24814 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:39:41.792168 24814 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:39:41.826408 24814 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.130:41397
I20260812 02:39:41.826458 24966 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.130:41397 every 8 connection(s)
I20260812 02:39:41.827766 24814 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-1/data/info.pb
I20260812 02:39:41.835780 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 24814
I20260812 02:39:41.835988 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-1/wal/instance
I20260812 02:39:41.836551 7698 external_mini_cluster.cc:1428] Setting key a71b6dffe8cf5d717478bf8df9dbb914
Aug 12 02:39:41 dist-test-slave-mn4r krb5kdc[24530](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502381, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.130@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 02:39:41.848428 24969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:41.830418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:34099 (local address 127.7.132.190:44921)
0812 02:39:41.830875 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0812 02:39:41.830881 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:39:41.831901 (+ 1020us) server_negotiation.cc:407] Connection header received
0812 02:39:41.833668 (+ 1767us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:39:41.833672 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:39:41.833741 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:39:41.833839 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:39:41.835385 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:41.836230 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:41.838323 (+ 2093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:41.838853 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:41.838987 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:39:41.842708 (+ 3721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:39:41.842713 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:39:41.842716 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:39:41.842747 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:39:41.845278 (+ 2531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:39:41.846024 (+ 746us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:39:41.846028 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:39:41.846030 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:39:41.846215 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:39:41.846659 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:39:41.846664 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:39:41.846668 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:39:41.846937 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:39:41.847104 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0812 02:39:41.847817 (+ 713us) server_negotiation.cc:299] Negotiation successful
0812 02:39:41.848051 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":332,"thread_start_us":205,"threads_started":1}
I20260812 02:39:41.849406 24971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:41.831039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:44921 (local address 127.7.132.130:34099)
0812 02:39:41.831740 (+ 701us) negotiation.cc:107] Waiting for socket to connect
0812 02:39:41.831780 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 02:39:41.833412 (+ 1632us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:39:41.834066 (+ 654us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:39:41.834126 (+ 60us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:39:41.834653 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:39:41.835223 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:39:41.835246 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:39:41.836393 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:39:41.836399 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:39:41.838178 (+ 1779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:39:41.838192 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:39:41.839103 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:39:41.839111 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:39:41.839346 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:39:41.840051 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:39:41.840073 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:39:41.842483 (+ 2410us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:39:41.845423 (+ 2940us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:39:41.845432 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:39:41.845449 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:39:41.845879 (+ 430us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:39:41.846328 (+ 449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:39:41.846332 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:39:41.846336 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:39:41.846536 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:39:41.847078 (+ 542us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:39:41.847087 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:39:41.847425 (+ 338us) client_negotiation.cc:769] Sending connection context
0812 02:39:41.847705 (+ 280us) client_negotiation.cc:240] Negotiation successful
0812 02:39:41.848220 (+ 515us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":263,"threads_started":1}
I20260812 02:39:41.850972 24967 heartbeater.cc:347] Connected to a master server at 127.7.132.190:44921
I20260812 02:39:41.851310 24967 heartbeater.cc:464] Registering TS with master...
I20260812 02:39:41.852167 24967 heartbeater.cc:511] Master 127.7.132.190:44921 requested a full tablet report, sending...
I20260812 02:39:41.853916 24585 ts_manager.cc:194] Registered new tserver with Master: 9a848619566249798e7b9f2d4118c664 (127.7.132.130:41397)
I20260812 02:39:41.855218 24585 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.130@KRBTEST.COM'} at 127.7.132.130:34099
WARNING: no policy specified for kudu/127.7.132.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.131.keytab.
Entry for principal kudu/127.7.132.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.131.keytab.
I20260812 02:39:41.901291 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.131:0
--local_ip_for_outbound_sockets=127.7.132.131
--webserver_interface=127.7.132.131
--webserver_port=0
--tserver_master_addrs=127.7.132.190:44921
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.131.keytab
--principal=kudu/127.7.132.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:42213
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:39:42.252581 24976 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:39:42.252928 24976 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:39:42.253123 24976 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:39:42.260370 24976 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:39:42.260538 24976 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:39:42.260756 24976 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.131
I20260812 02:39:42.270706 24976 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:42213
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.131.keytab
--principal=kudu/127.7.132.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.132.131
--webserver_port=0
--tserver_master_addrs=127.7.132.190:44921
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.131
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:39:42.272094 24976 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:39:42.273444 24976 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:39:42.287336 24984 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:39:42.287511 24986 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:39:42.287441 24983 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:39:42.288517 24976 server_base.cc:1034] running on GCE node
I20260812 02:39:42.289131 24976 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:39:42.290037 24976 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:39:42.291289 24976 hybrid_clock.cc:648] HybridClock initialized: now 1786502382291238 us; error 39 us; skew 500 ppm
I20260812 02:39:42.291625 24976 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:39:42 dist-test-slave-mn4r krb5kdc[24530](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502382, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:39:42.294885 24976 init.cc:377] Logged in from keytab as kudu/127.7.132.131@KRBTEST.COM (short username kudu)
I20260812 02:39:42.296346 24976 webserver.cc:460] Webserver started at http://127.7.132.131:41163/ using document root <none> and password file <none>
I20260812 02:39:42.297066 24976 fs_manager.cc:359] Metadata directory not provided
I20260812 02:39:42.297214 24976 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:39:42.297529 24976 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:39:42.300681 24976 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-2/data/instance:
uuid: "7c0a7ebfc3e74adeb520daaaf7609f27"
format_stamp: "Formatted at 2026-08-12 02:39:42 on dist-test-slave-mn4r"
server_key: "82d319a99df0be3d1797d77d9546b6aa"
server_key_iv: "763557f03a3893646fb09d441e17cf97"
server_key_version: "encryptionkey@0"
I20260812 02:39:42.301451 24976 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-2/wal/instance:
uuid: "7c0a7ebfc3e74adeb520daaaf7609f27"
format_stamp: "Formatted at 2026-08-12 02:39:42 on dist-test-slave-mn4r"
server_key: "82d319a99df0be3d1797d77d9546b6aa"
server_key_iv: "763557f03a3893646fb09d441e17cf97"
server_key_version: "encryptionkey@0"
I20260812 02:39:42.306963 24976 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260812 02:39:42.310735 24996 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:39:42.312099 24976 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 02:39:42.312284 24976 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-2/wal
uuid: "7c0a7ebfc3e74adeb520daaaf7609f27"
format_stamp: "Formatted at 2026-08-12 02:39:42 on dist-test-slave-mn4r"
server_key: "82d319a99df0be3d1797d77d9546b6aa"
server_key_iv: "763557f03a3893646fb09d441e17cf97"
server_key_version: "encryptionkey@0"
I20260812 02:39:42.312501 24976 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:39:42.317538 24806 heartbeater.cc:503] Master 127.7.132.190:44921 was elected leader, sending a full tablet report...
I20260812 02:39:42.326745 24976 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:39:42.330818 24976 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:39:42.331095 24976 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:39:42.331972 24976 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:39:42.333280 24976 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:39:42.333355 24976 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:39:42.333449 24976 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:39:42.333495 24976 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:39:42.367332 24976 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.131:46073
I20260812 02:39:42.367379 25119 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.131:46073 every 8 connection(s)
I20260812 02:39:42.368726 24976 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-2/data/info.pb
I20260812 02:39:42.369926 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 24976
I20260812 02:39:42.370177 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-2/wal/instance
I20260812 02:39:42.370820 7698 external_mini_cluster.cc:1428] Setting key a8f93383b7da94173dbdfd57bf6c9c80
Aug 12 02:39:42 dist-test-slave-mn4r krb5kdc[24530](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502382, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.131@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 02:39:42.390198 25123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:42.372017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:35507 (local address 127.7.132.190:44921)
0812 02:39:42.372537 (+ 520us) server_negotiation.cc:206] Beginning negotiation
0812 02:39:42.372542 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:39:42.373663 (+ 1121us) server_negotiation.cc:407] Connection header received
0812 02:39:42.375616 (+ 1953us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:39:42.375621 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:39:42.375715 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:39:42.375830 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:39:42.377340 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:42.378218 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:42.379766 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:42.380174 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:42.380311 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:39:42.383813 (+ 3502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:39:42.383818 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:39:42.383821 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:39:42.383853 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:39:42.387328 (+ 3475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:39:42.388079 (+ 751us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:39:42.388083 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:39:42.388086 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:39:42.388204 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:39:42.388586 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:39:42.388590 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:39:42.388592 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:39:42.388904 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:39:42.389039 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0812 02:39:42.389690 (+ 651us) server_negotiation.cc:299] Negotiation successful
0812 02:39:42.389859 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":393,"thread_start_us":243,"threads_started":1}
I20260812 02:39:42.391204 25124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:42.372725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:44921 (local address 127.7.132.131:35507)
0812 02:39:42.373515 (+ 790us) negotiation.cc:107] Waiting for socket to connect
0812 02:39:42.373554 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 02:39:42.375341 (+ 1787us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:39:42.376088 (+ 747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:39:42.376097 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:39:42.376603 (+ 506us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:39:42.377173 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:39:42.377190 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:39:42.378372 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:39:42.378376 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:39:42.379613 (+ 1237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:39:42.379626 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:39:42.380412 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:39:42.380420 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:39:42.380647 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:39:42.381336 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:39:42.381358 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:39:42.383612 (+ 2254us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:39:42.387463 (+ 3851us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:39:42.387470 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:39:42.387495 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:39:42.387965 (+ 470us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:39:42.388294 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:39:42.388297 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:39:42.388299 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:39:42.388474 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:39:42.389027 (+ 553us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:39:42.389033 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:39:42.389329 (+ 296us) client_negotiation.cc:769] Sending connection context
0812 02:39:42.389567 (+ 238us) client_negotiation.cc:240] Negotiation successful
0812 02:39:42.389953 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":547,"thread_start_us":302,"threads_started":1}
I20260812 02:39:42.392609 25120 heartbeater.cc:347] Connected to a master server at 127.7.132.190:44921
I20260812 02:39:42.392907 25120 heartbeater.cc:464] Registering TS with master...
I20260812 02:39:42.393622 25120 heartbeater.cc:511] Master 127.7.132.190:44921 requested a full tablet report, sending...
I20260812 02:39:42.395022 24585 ts_manager.cc:194] Registered new tserver with Master: 7c0a7ebfc3e74adeb520daaaf7609f27 (127.7.132.131:46073)
I20260812 02:39:42.395805 24585 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.131@KRBTEST.COM'} at 127.7.132.131:35507
I20260812 02:39:42.399191 7698 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 02:39:42.416204 25123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:42.403709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42714 (local address 127.7.132.190:44921)
0812 02:39:42.403874 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0812 02:39:42.403878 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 02:39:42.404386 (+ 508us) server_negotiation.cc:407] Connection header received
0812 02:39:42.404552 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:39:42.404557 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:39:42.404617 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:39:42.404762 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:39:42.406299 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:42.407072 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:42.408333 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:42.408811 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:42.408903 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:39:42.411321 (+ 2418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:39:42.411326 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:39:42.411329 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:39:42.411358 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:39:42.413925 (+ 2567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:39:42.414534 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:39:42.414538 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:39:42.414540 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:39:42.414655 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:39:42.414978 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:39:42.414981 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:39:42.414984 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:39:42.415302 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:39:42.415487 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0812 02:39:42.415754 (+ 267us) server_negotiation.cc:299] Negotiation successful
0812 02:39:42.415917 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 02:39:42.420717 24585 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42714:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 02:39:42.427644 24585 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 02:39:42.454298 25135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:42.440305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.130:41397 (local address 127.0.0.1:55638)
0812 02:39:42.441830 (+ 1525us) negotiation.cc:107] Waiting for socket to connect
0812 02:39:42.441897 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0812 02:39:42.442139 (+ 242us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:39:42.446132 (+ 3993us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:39:42.446140 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:39:42.446173 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:39:42.446967 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:39:42.446982 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:39:42.448853 (+ 1871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:39:42.448860 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:39:42.451339 (+ 2479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:39:42.451353 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:39:42.453213 (+ 1860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:39:42.453219 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:39:42.453408 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:39:42.453460 (+ 52us) client_negotiation.cc:769] Sending connection context
0812 02:39:42.453603 (+ 143us) client_negotiation.cc:240] Negotiation successful
0812 02:39:42.453801 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1180,"mutex_wait_us":3,"thread_start_us":5199,"threads_started":1}
I20260812 02:39:42.454334 25138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:42.440392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55638 (local address 127.7.132.130:41397)
0812 02:39:42.445401 (+ 5009us) server_negotiation.cc:206] Beginning negotiation
0812 02:39:42.445411 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 02:39:42.445446 (+ 35us) server_negotiation.cc:407] Connection header received
0812 02:39:42.445578 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:39:42.445587 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:39:42.445865 (+ 278us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:39:42.446623 (+ 758us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:39:42.447177 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:42.448722 (+ 1545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:42.451519 (+ 2797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:42.452981 (+ 1462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:42.453130 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:39:42.453322 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0812 02:39:42.453874 (+ 552us) server_negotiation.cc:299] Negotiation successful
0812 02:39:42.454037 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4874,"thread_start_us":4783,"threads_started":1}
I20260812 02:39:42.454684 25139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:42.441107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40120 (local address 127.7.132.129:41103)
0812 02:39:42.444982 (+ 3875us) server_negotiation.cc:206] Beginning negotiation
0812 02:39:42.444995 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 02:39:42.446043 (+ 1048us) server_negotiation.cc:407] Connection header received
0812 02:39:42.446186 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:39:42.446193 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:39:42.446423 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:39:42.446636 (+ 213us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:39:42.447708 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:42.449173 (+ 1465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:42.451808 (+ 2635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:42.453315 (+ 1507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:42.453493 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:39:42.453734 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0812 02:39:42.454195 (+ 461us) server_negotiation.cc:299] Negotiation successful
0812 02:39:42.454362 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3648,"thread_start_us":3452,"threads_started":1}
I20260812 02:39:42.455641 25142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:42.444247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56762 (local address 127.7.132.131:46073)
0812 02:39:42.446229 (+ 1982us) server_negotiation.cc:206] Beginning negotiation
0812 02:39:42.446240 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 02:39:42.447366 (+ 1126us) server_negotiation.cc:407] Connection header received
0812 02:39:42.447497 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:39:42.447504 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:39:42.447715 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:39:42.447913 (+ 198us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:39:42.448596 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:42.449870 (+ 1274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:42.452579 (+ 2709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:42.453774 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:42.453892 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:39:42.454148 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0812 02:39:42.455282 (+ 1134us) server_negotiation.cc:299] Negotiation successful
0812 02:39:42.455404 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1765,"thread_start_us":1609,"threads_started":1}
I20260812 02:39:42.455911 25141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:42.443466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.131:46073 (local address 127.0.0.1:56762)
0812 02:39:42.447083 (+ 3617us) negotiation.cc:107] Waiting for socket to connect
0812 02:39:42.447110 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0812 02:39:42.447220 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:39:42.448023 (+ 803us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:39:42.448027 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:39:42.448064 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:39:42.448412 (+ 348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:39:42.448423 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:39:42.450022 (+ 1599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:39:42.450027 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:39:42.452428 (+ 2401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:39:42.452440 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:39:42.454304 (+ 1864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:39:42.454313 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:39:42.454513 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:39:42.454542 (+ 29us) client_negotiation.cc:769] Sending connection context
0812 02:39:42.454625 (+ 83us) client_negotiation.cc:240] Negotiation successful
0812 02:39:42.454741 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3486,"mutex_wait_us":663,"thread_start_us":2995,"threads_started":1}
I20260812 02:39:42.455948 25136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:42.440377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:41103 (local address 127.0.0.1:40120)
0812 02:39:42.445769 (+ 5392us) negotiation.cc:107] Waiting for socket to connect
0812 02:39:42.445788 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 02:39:42.445914 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:39:42.446627 (+ 713us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:39:42.446632 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:39:42.446667 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:39:42.447511 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:39:42.447525 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:39:42.449352 (+ 1827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:39:42.449358 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:39:42.451644 (+ 2286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:39:42.451658 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:39:42.453552 (+ 1894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:39:42.453559 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:39:42.453737 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:39:42.453783 (+ 46us) client_negotiation.cc:769] Sending connection context
0812 02:39:42.453870 (+ 87us) client_negotiation.cc:240] Negotiation successful
0812 02:39:42.454016 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5278,"mutex_wait_us":1,"thread_start_us":935,"threads_started":1}
I20260812 02:39:42.459144 24891 tablet_service.cc:1467] Processing CreateTablet for tablet 461eaa3ddc9a4a66b8dd7e8041bf0f22 (DEFAULT_TABLE table=test-table [id=5646291aa1764707befe564d8567722b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:39:42.459935 24730 tablet_service.cc:1467] Processing CreateTablet for tablet 461eaa3ddc9a4a66b8dd7e8041bf0f22 (DEFAULT_TABLE table=test-table [id=5646291aa1764707befe564d8567722b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:39:42.459976 25048 tablet_service.cc:1467] Processing CreateTablet for tablet 461eaa3ddc9a4a66b8dd7e8041bf0f22 (DEFAULT_TABLE table=test-table [id=5646291aa1764707befe564d8567722b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:39:42.460736 24891 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 461eaa3ddc9a4a66b8dd7e8041bf0f22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:39:42.461184 24730 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 461eaa3ddc9a4a66b8dd7e8041bf0f22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:39:42.461390 25048 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 461eaa3ddc9a4a66b8dd7e8041bf0f22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:39:42.473452 25144 tablet_bootstrap.cc:492] T 461eaa3ddc9a4a66b8dd7e8041bf0f22 P a78f88301ed54f48bf9e17f58fd62d5e: Bootstrap starting.
I20260812 02:39:42.476030 25147 tablet_bootstrap.cc:492] T 461eaa3ddc9a4a66b8dd7e8041bf0f22 P 7c0a7ebfc3e74adeb520daaaf7609f27: Bootstrap starting.
I20260812 02:39:42.477335 25144 tablet_bootstrap.cc:654] T 461eaa3ddc9a4a66b8dd7e8041bf0f22 P a78f88301ed54f48bf9e17f58fd62d5e: Neither blocks nor log segments found. Creating new log.
I20260812 02:39:42.478727 25144 log.cc:824] T 461eaa3ddc9a4a66b8dd7e8041bf0f22 P a78f88301ed54f48bf9e17f58fd62d5e: Log is configured to *not* fsync() on all Append() calls
I20260812 02:39:42.479724 25147 tablet_bootstrap.cc:654] T 461eaa3ddc9a4a66b8dd7e8041bf0f22 P 7c0a7ebfc3e74adeb520daaaf7609f27: Neither blocks nor log segments found. Creating new log.
I20260812 02:39:42.481354 25147 log.cc:824] T 461eaa3ddc9a4a66b8dd7e8041bf0f22 P 7c0a7ebfc3e74adeb520daaaf7609f27: Log is configured to *not* fsync() on all Append() calls
I20260812 02:39:42.482888 25145 tablet_bootstrap.cc:492] T 461eaa3ddc9a4a66b8dd7e8041bf0f22 P 9a848619566249798e7b9f2d4118c664: Bootstrap starting.
I20260812 02:39:42.487500 25145 tablet_bootstrap.cc:654] T 461eaa3ddc9a4a66b8dd7e8041bf0f22 P 9a848619566249798e7b9f2d4118c664: Neither blocks nor log segments found. Creating new log.
I20260812 02:39:42.488857 25145 log.cc:824] T 461eaa3ddc9a4a66b8dd7e8041bf0f22 P 9a848619566249798e7b9f2d4118c664: Log is configured to *not* fsync() on all Append() calls
I20260812 02:39:42.488946 25144 tablet_bootstrap.cc:492] T 461eaa3ddc9a4a66b8dd7e8041bf0f22 P a78f88301ed54f48bf9e17f58fd62d5e: No bootstrap required, opened a new log
I20260812 02:39:42.489110 25147 tablet_bootstrap.cc:492] T 461eaa3ddc9a4a66b8dd7e8041bf0f22 P 7c0a7ebfc3e74adeb520daaaf7609f27: No bootstrap required, opened a new log
I20260812 02:39:42.489277 25144 ts_tablet_manager.cc:1397] T 461eaa3ddc9a4a66b8dd7e8041bf0f22 P a78f88301ed54f48bf9e17f58fd62d5e: Time spent bootstrapping tablet: real 0.016s user 0.010s sys 0.004s
I20260812 02:39:42.489392 25147 ts_tablet_manager.cc:1397] T 461eaa3ddc9a4a66b8dd7e8041bf0f22 P 7c0a7ebfc3e74adeb520daaaf7609f27: Time spent bootstrapping tablet: real 0.014s user 0.007s sys 0.000s
I20260812 02:39:42.493852 25145 tablet_bootstrap.cc:492] T 461eaa3ddc9a4a66b8dd7e8041bf0f22 P 9a848619566249798e7b9f2d4118c664: No bootstrap required, opened a new log
I20260812 02:39:42.494207 25145 ts_tablet_manager.cc:1397] T 461eaa3ddc9a4a66b8dd7e8041bf0f22 P 9a848619566249798e7b9f2d4118c664: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.006s
I20260812 02:39:42.498190 25144 raft_consensus.cc:348] T 461eaa3ddc9a4a66b8dd7e8041bf0f22 P a78f88301ed54f48bf9e17f58fd62d5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a848619566249798e7b9f2d4118c664" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 41397 } } peers { permanent_uuid: "a78f88301ed54f48bf9e17f58fd62d5e" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 41103 } } peers { permanent_uuid: "7c0a7ebfc3e74adeb520daaaf7609f27" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 46073 } }
I20260812 02:39:42.498721 25144 raft_consensus.cc:374] T 461eaa3ddc9a4a66b8dd7e8041bf0f22 P a78f88301ed54f48bf9e17f58fd62d5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:39:42.498832 25144 raft_consensus.cc:729] T 461eaa3ddc9a4a66b8dd7e8041bf0f22 P a78f88301ed54f48bf9e17f58fd62d5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a78f88301ed54f48bf9e17f58fd62d5e, State: Initialized, Role: FOLLOWER
I20260812 02:39:42.499715 25144 consensus_queue.cc:260] T 461eaa3ddc9a4a66b8dd7e8041bf0f22 P a78f88301ed54f48bf9e17f58fd62d5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a848619566249798e7b9f2d4118c664" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 41397 } } peers { permanent_uuid: "a78f88301ed54f48bf9e17f58fd62d5e" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 41103 } } peers { permanent_uuid: "7c0a7ebfc3e74adeb520daaaf7609f27" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 46073 } }
I20260812 02:39:42.500505 25147 raft_consensus.cc:348] T 461eaa3ddc9a4a66b8dd7e8041bf0f22 P 7c0a7ebfc3e74adeb520daaaf7609f27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a848619566249798e7b9f2d4118c664" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 41397 } } peers { permanent_uuid: "a78f88301ed54f48bf9e17f58fd62d5e" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 41103 } } peers { permanent_uuid: "7c0a7ebfc3e74adeb520daaaf7609f27" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 46073 } }
I20260812 02:39:42.500985 25147 raft_consensus.cc:374] T 461eaa3ddc9a4a66b8dd7e8041bf0f22 P 7c0a7ebfc3e74adeb520daaaf7609f27 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:39:42.501089 25147 raft_consensus.cc:729] T 461eaa3ddc9a4a66b8dd7e8041bf0f22 P 7c0a7ebfc3e74adeb520daaaf7609f27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c0a7ebfc3e74adeb520daaaf7609f27, State: Initialized, Role: FOLLOWER
I20260812 02:39:42.501873 25147 consensus_queue.cc:260] T 461eaa3ddc9a4a66b8dd7e8041bf0f22 P 7c0a7ebfc3e74adeb520daaaf7609f27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a848619566249798e7b9f2d4118c664" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 41397 } } peers { permanent_uuid: "a78f88301ed54f48bf9e17f58fd62d5e" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 41103 } } peers { permanent_uuid: "7c0a7ebfc3e74adeb520daaaf7609f27" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 46073 } }
I20260812 02:39:42.506927 25145 raft_consensus.cc:348] T 461eaa3ddc9a4a66b8dd7e8041bf0f22 P 9a848619566249798e7b9f2d4118c664 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a848619566249798e7b9f2d4118c664" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 41397 } } peers { permanent_uuid: "a78f88301ed54f48bf9e17f58fd62d5e" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 41103 } } peers { permanent_uuid: "7c0a7ebfc3e74adeb520daaaf7609f27" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 46073 } }
I20260812 02:39:42.507718 25145 raft_consensus.cc:374] T 461eaa3ddc9a4a66b8dd7e8041bf0f22 P 9a848619566249798e7b9f2d4118c664 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:39:42.507840 25145 raft_consensus.cc:729] T 461eaa3ddc9a4a66b8dd7e8041bf0f22 P 9a848619566249798e7b9f2d4118c664 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a848619566249798e7b9f2d4118c664, State: Initialized, Role: FOLLOWER
I20260812 02:39:42.508358 25144 ts_tablet_manager.cc:1428] T 461eaa3ddc9a4a66b8dd7e8041bf0f22 P a78f88301ed54f48bf9e17f58fd62d5e: Time spent starting tablet: real 0.019s user 0.011s sys 0.007s
I20260812 02:39:42.508805 25145 consensus_queue.cc:260] T 461eaa3ddc9a4a66b8dd7e8041bf0f22 P 9a848619566249798e7b9f2d4118c664 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a848619566249798e7b9f2d4118c664" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 41397 } } peers { permanent_uuid: "a78f88301ed54f48bf9e17f58fd62d5e" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 41103 } } peers { permanent_uuid: "7c0a7ebfc3e74adeb520daaaf7609f27" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 46073 } }
I20260812 02:39:42.509919 25147 ts_tablet_manager.cc:1428] T 461eaa3ddc9a4a66b8dd7e8041bf0f22 P 7c0a7ebfc3e74adeb520daaaf7609f27: Time spent starting tablet: real 0.020s user 0.011s sys 0.009s
I20260812 02:39:42.510543 25120 heartbeater.cc:503] Master 127.7.132.190:44921 was elected leader, sending a full tablet report...
I20260812 02:39:42.513985 24967 heartbeater.cc:503] Master 127.7.132.190:44921 was elected leader, sending a full tablet report...
I20260812 02:39:42.517282 25145 ts_tablet_manager.cc:1428] T 461eaa3ddc9a4a66b8dd7e8041bf0f22 P 9a848619566249798e7b9f2d4118c664: Time spent starting tablet: real 0.023s user 0.014s sys 0.006s
W20260812 02:39:42.534457 24807 tablet.cc:2369] T 461eaa3ddc9a4a66b8dd7e8041bf0f22 P a78f88301ed54f48bf9e17f58fd62d5e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 02:39:42.554198 25153 raft_consensus.cc:482] T 461eaa3ddc9a4a66b8dd7e8041bf0f22 P 7c0a7ebfc3e74adeb520daaaf7609f27 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 02:39:42.554557 25153 raft_consensus.cc:504] T 461eaa3ddc9a4a66b8dd7e8041bf0f22 P 7c0a7ebfc3e74adeb520daaaf7609f27 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a848619566249798e7b9f2d4118c664" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 41397 } } peers { permanent_uuid: "a78f88301ed54f48bf9e17f58fd62d5e" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 41103 } } peers { permanent_uuid: "7c0a7ebfc3e74adeb520daaaf7609f27" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 46073 } }
I20260812 02:39:42.556720 25153 leader_election.cc:288] T 461eaa3ddc9a4a66b8dd7e8041bf0f22 P 7c0a7ebfc3e74adeb520daaaf7609f27 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9a848619566249798e7b9f2d4118c664 (127.7.132.130:41397), a78f88301ed54f48bf9e17f58fd62d5e (127.7.132.129:41103)
I20260812 02:39:42.563099 25138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:42.557308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:51739 (local address 127.7.132.130:41397)
0812 02:39:42.557541 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0812 02:39:42.557548 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:39:42.557566 (+ 18us) server_negotiation.cc:407] Connection header received
0812 02:39:42.557634 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:39:42.557639 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:39:42.557707 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:39:42.557798 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:39:42.558301 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:42.559410 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:42.561303 (+ 1893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:42.562168 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:42.562337 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:39:42.562414 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 02:39:42.562706 (+ 292us) server_negotiation.cc:299] Negotiation successful
0812 02:39:42.562839 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260812 02:39:42.563098 25124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:42.556970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.130:41397 (local address 127.7.132.131:51739)
0812 02:39:42.557224 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0812 02:39:42.557246 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 02:39:42.557383 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:39:42.557821 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:39:42.557826 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:39:42.557863 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:39:42.558126 (+ 263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:39:42.558140 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:39:42.559590 (+ 1450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:39:42.559596 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:39:42.561135 (+ 1539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:39:42.561147 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:39:42.562337 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:39:42.562343 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:39:42.562529 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:39:42.562561 (+ 32us) client_negotiation.cc:769] Sending connection context
0812 02:39:42.562668 (+ 107us) client_negotiation.cc:240] Negotiation successful
0812 02:39:42.562804 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63,"mutex_wait_us":1}
I20260812 02:39:42.564342 24914 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "461eaa3ddc9a4a66b8dd7e8041bf0f22" candidate_uuid: "7c0a7ebfc3e74adeb520daaaf7609f27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a848619566249798e7b9f2d4118c664" is_pre_election: true
I20260812 02:39:42.564878 24914 raft_consensus.cc:2454] T 461eaa3ddc9a4a66b8dd7e8041bf0f22 P 9a848619566249798e7b9f2d4118c664 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c0a7ebfc3e74adeb520daaaf7609f27 in term 0.
I20260812 02:39:42.565260 25139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:42.557448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:41467 (local address 127.7.132.129:41103)
0812 02:39:42.558128 (+ 680us) server_negotiation.cc:206] Beginning negotiation
0812 02:39:42.558134 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:39:42.558310 (+ 176us) server_negotiation.cc:407] Connection header received
0812 02:39:42.558517 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:39:42.558521 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:39:42.558580 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:39:42.558682 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:39:42.559202 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:42.560207 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:42.563005 (+ 2798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:42.564248 (+ 1243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:42.564404 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:39:42.564469 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 02:39:42.564918 (+ 449us) server_negotiation.cc:299] Negotiation successful
0812 02:39:42.565043 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":483}
I20260812 02:39:42.565629 24997 leader_election.cc:302] T 461eaa3ddc9a4a66b8dd7e8041bf0f22 P 7c0a7ebfc3e74adeb520daaaf7609f27 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c0a7ebfc3e74adeb520daaaf7609f27, 9a848619566249798e7b9f2d4118c664; no voters:
I20260812 02:39:42.566071 25153 raft_consensus.cc:2781] T 461eaa3ddc9a4a66b8dd7e8041bf0f22 P 7c0a7ebfc3e74adeb520daaaf7609f27 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 02:39:42.566233 25153 raft_consensus.cc:482] T 461eaa3ddc9a4a66b8dd7e8041bf0f22 P 7c0a7ebfc3e74adeb520daaaf7609f27 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 02:39:42.566363 25153 raft_consensus.cc:3037] T 461eaa3ddc9a4a66b8dd7e8041bf0f22 P 7c0a7ebfc3e74adeb520daaaf7609f27 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:39:42.570978 25153 raft_consensus.cc:504] T 461eaa3ddc9a4a66b8dd7e8041bf0f22 P 7c0a7ebfc3e74adeb520daaaf7609f27 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a848619566249798e7b9f2d4118c664" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 41397 } } peers { permanent_uuid: "a78f88301ed54f48bf9e17f58fd62d5e" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 41103 } } peers { permanent_uuid: "7c0a7ebfc3e74adeb520daaaf7609f27" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 46073 } }
I20260812 02:39:42.572002 24914 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "461eaa3ddc9a4a66b8dd7e8041bf0f22" candidate_uuid: "7c0a7ebfc3e74adeb520daaaf7609f27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a848619566249798e7b9f2d4118c664"
I20260812 02:39:42.572221 24914 raft_consensus.cc:3037] T 461eaa3ddc9a4a66b8dd7e8041bf0f22 P 9a848619566249798e7b9f2d4118c664 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:39:42.572892 25156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:42.557105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:41103 (local address 127.7.132.131:41467)
0812 02:39:42.558200 (+ 1095us) negotiation.cc:107] Waiting for socket to connect
0812 02:39:42.558216 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 02:39:42.558361 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:39:42.558700 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:39:42.558704 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:39:42.558726 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:39:42.559041 (+ 315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:39:42.559052 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:39:42.560373 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:39:42.560379 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:39:42.562797 (+ 2418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:39:42.562815 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:39:42.564605 (+ 1790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:39:42.564611 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:39:42.564755 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:39:42.564783 (+ 28us) client_negotiation.cc:769] Sending connection context
0812 02:39:42.565465 (+ 682us) client_negotiation.cc:240] Negotiation successful
0812 02:39:42.565618 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":928,"thread_start_us":688,"threads_started":1}
I20260812 02:39:42.573480 25153 leader_election.cc:288] T 461eaa3ddc9a4a66b8dd7e8041bf0f22 P 7c0a7ebfc3e74adeb520daaaf7609f27 [CANDIDATE]: Term 1 election: Requested vote from peers 9a848619566249798e7b9f2d4118c664 (127.7.132.130:41397), a78f88301ed54f48bf9e17f58fd62d5e (127.7.132.129:41103)
I20260812 02:39:42.574105 24754 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "461eaa3ddc9a4a66b8dd7e8041bf0f22" candidate_uuid: "7c0a7ebfc3e74adeb520daaaf7609f27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a78f88301ed54f48bf9e17f58fd62d5e" is_pre_election: true
I20260812 02:39:42.574709 24754 raft_consensus.cc:2454] T 461eaa3ddc9a4a66b8dd7e8041bf0f22 P a78f88301ed54f48bf9e17f58fd62d5e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c0a7ebfc3e74adeb520daaaf7609f27 in term 0.
I20260812 02:39:42.575228 24753 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "461eaa3ddc9a4a66b8dd7e8041bf0f22" candidate_uuid: "7c0a7ebfc3e74adeb520daaaf7609f27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a78f88301ed54f48bf9e17f58fd62d5e"
I20260812 02:39:42.575484 24753 raft_consensus.cc:3037] T 461eaa3ddc9a4a66b8dd7e8041bf0f22 P a78f88301ed54f48bf9e17f58fd62d5e [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:39:42.575613 24914 raft_consensus.cc:2454] T 461eaa3ddc9a4a66b8dd7e8041bf0f22 P 9a848619566249798e7b9f2d4118c664 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c0a7ebfc3e74adeb520daaaf7609f27 in term 1.
I20260812 02:39:42.576061 24997 leader_election.cc:302] T 461eaa3ddc9a4a66b8dd7e8041bf0f22 P 7c0a7ebfc3e74adeb520daaaf7609f27 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c0a7ebfc3e74adeb520daaaf7609f27, 9a848619566249798e7b9f2d4118c664; no voters:
I20260812 02:39:42.576363 25153 raft_consensus.cc:2781] T 461eaa3ddc9a4a66b8dd7e8041bf0f22 P 7c0a7ebfc3e74adeb520daaaf7609f27 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:39:42.576936 25153 raft_consensus.cc:686] T 461eaa3ddc9a4a66b8dd7e8041bf0f22 P 7c0a7ebfc3e74adeb520daaaf7609f27 [term 1 LEADER]: Becoming Leader. State: Replica: 7c0a7ebfc3e74adeb520daaaf7609f27, State: Running, Role: LEADER
I20260812 02:39:42.577556 25153 consensus_queue.cc:237] T 461eaa3ddc9a4a66b8dd7e8041bf0f22 P 7c0a7ebfc3e74adeb520daaaf7609f27 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a848619566249798e7b9f2d4118c664" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 41397 } } peers { permanent_uuid: "a78f88301ed54f48bf9e17f58fd62d5e" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 41103 } } peers { permanent_uuid: "7c0a7ebfc3e74adeb520daaaf7609f27" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 46073 } }
I20260812 02:39:42.578886 24753 raft_consensus.cc:2454] T 461eaa3ddc9a4a66b8dd7e8041bf0f22 P a78f88301ed54f48bf9e17f58fd62d5e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c0a7ebfc3e74adeb520daaaf7609f27 in term 1.
W20260812 02:39:42.579828 24968 tablet.cc:2369] T 461eaa3ddc9a4a66b8dd7e8041bf0f22 P 9a848619566249798e7b9f2d4118c664: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 02:39:42.583032 24583 catalog_manager.cc:5495] T 461eaa3ddc9a4a66b8dd7e8041bf0f22 P 7c0a7ebfc3e74adeb520daaaf7609f27 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c0a7ebfc3e74adeb520daaaf7609f27 (127.7.132.131). New cstate: current_term: 1 leader_uuid: "7c0a7ebfc3e74adeb520daaaf7609f27" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a848619566249798e7b9f2d4118c664" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 41397 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a78f88301ed54f48bf9e17f58fd62d5e" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 41103 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7c0a7ebfc3e74adeb520daaaf7609f27" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 46073 } health_report { overall_health: HEALTHY } } }
I20260812 02:39:42.612968 25142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:42.606192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56764 (local address 127.7.132.131:46073)
0812 02:39:42.606419 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0812 02:39:42.606424 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:39:42.606442 (+ 18us) server_negotiation.cc:407] Connection header received
0812 02:39:42.606667 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:39:42.606673 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:39:42.606791 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:39:42.606961 (+ 170us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 02:39:42.607368 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:42.608671 (+ 1303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:42.610222 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:42.610949 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:42.611093 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:39:42.611595 (+ 502us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 02:39:42.612315 (+ 720us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 02:39:42.612473 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0812 02:39:42.612532 (+ 59us) server_negotiation.cc:299] Negotiation successful
0812 02:39:42.612639 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
W20260812 02:39:42.621193 25122 tablet.cc:2369] T 461eaa3ddc9a4a66b8dd7e8041bf0f22 P 7c0a7ebfc3e74adeb520daaaf7609f27: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 02:39:42.622192 24753 raft_consensus.cc:1261] T 461eaa3ddc9a4a66b8dd7e8041bf0f22 P a78f88301ed54f48bf9e17f58fd62d5e [term 1 FOLLOWER]: Refusing update from remote peer 7c0a7ebfc3e74adeb520daaaf7609f27: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:39:42.622344 24914 raft_consensus.cc:1261] T 461eaa3ddc9a4a66b8dd7e8041bf0f22 P 9a848619566249798e7b9f2d4118c664 [term 1 FOLLOWER]: Refusing update from remote peer 7c0a7ebfc3e74adeb520daaaf7609f27: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:39:42.623265 25157 consensus_queue.cc:1035] T 461eaa3ddc9a4a66b8dd7e8041bf0f22 P 7c0a7ebfc3e74adeb520daaaf7609f27 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a78f88301ed54f48bf9e17f58fd62d5e" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 41103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:39:42.623610 25153 consensus_queue.cc:1035] T 461eaa3ddc9a4a66b8dd7e8041bf0f22 P 7c0a7ebfc3e74adeb520daaaf7609f27 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a848619566249798e7b9f2d4118c664" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 41397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:39:42.635901 25160 mvcc.cc:205] Tried to move back new op lower bound from 7317513759209799680 to 7317513759042969600. Current Snapshot: MvccSnapshot[applied={T|T < 7317513759209799680}]
I20260812 02:39:42.649567 24583 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42714:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 02:39:42.649879 24583 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42714:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 02:39:42.653610 24583 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ce6e17ddb54e4dc39dc6f8ef9c7f2d94: Sending DeleteTablet for 3 replicas of tablet 461eaa3ddc9a4a66b8dd7e8041bf0f22
I20260812 02:39:42.655030 24891 tablet_service.cc:1514] Processing DeleteTablet for tablet 461eaa3ddc9a4a66b8dd7e8041bf0f22 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:39:42 UTC) from {username='kudu', principal='kudu/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:55638
I20260812 02:39:42.655840 25048 tablet_service.cc:1514] Processing DeleteTablet for tablet 461eaa3ddc9a4a66b8dd7e8041bf0f22 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:39:42 UTC) from {username='kudu', principal='kudu/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:56762
I20260812 02:39:42.656126 25170 tablet_replica.cc:330] T 461eaa3ddc9a4a66b8dd7e8041bf0f22 P 9a848619566249798e7b9f2d4118c664: stopping tablet replica
I20260812 02:39:42.656625 25170 raft_consensus.cc:2229] T 461eaa3ddc9a4a66b8dd7e8041bf0f22 P 9a848619566249798e7b9f2d4118c664 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 02:39:42.656849 24730 tablet_service.cc:1514] Processing DeleteTablet for tablet 461eaa3ddc9a4a66b8dd7e8041bf0f22 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:39:42 UTC) from {username='kudu', principal='kudu/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:40120
I20260812 02:39:42.657356 25170 raft_consensus.cc:2258] T 461eaa3ddc9a4a66b8dd7e8041bf0f22 P 9a848619566249798e7b9f2d4118c664 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:39:42.659757 25170 ts_tablet_manager.cc:1905] T 461eaa3ddc9a4a66b8dd7e8041bf0f22 P 9a848619566249798e7b9f2d4118c664: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:39:42.660138 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 24658
I20260812 02:39:42.668530 25170 ts_tablet_manager.cc:1918] T 461eaa3ddc9a4a66b8dd7e8041bf0f22 P 9a848619566249798e7b9f2d4118c664: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 02:39:42.668850 25170 log.cc:1196] T 461eaa3ddc9a4a66b8dd7e8041bf0f22 P 9a848619566249798e7b9f2d4118c664: Deleting WAL directory at /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-1/wal/wals/461eaa3ddc9a4a66b8dd7e8041bf0f22
I20260812 02:39:42.669555 25170 ts_tablet_manager.cc:1939] T 461eaa3ddc9a4a66b8dd7e8041bf0f22 P 9a848619566249798e7b9f2d4118c664: Deleting consensus metadata
I20260812 02:39:42.671896 24567 catalog_manager.cc:4841] TS 9a848619566249798e7b9f2d4118c664 (127.7.132.130:41397): tablet 461eaa3ddc9a4a66b8dd7e8041bf0f22 (table test-table [id=5646291aa1764707befe564d8567722b]) successfully deleted
I20260812 02:39:42.675122 25172 tablet_replica.cc:330] T 461eaa3ddc9a4a66b8dd7e8041bf0f22 P a78f88301ed54f48bf9e17f58fd62d5e: stopping tablet replica
I20260812 02:39:42.675663 25172 raft_consensus.cc:2229] T 461eaa3ddc9a4a66b8dd7e8041bf0f22 P a78f88301ed54f48bf9e17f58fd62d5e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 02:39:42.676292 25172 raft_consensus.cc:2258] T 461eaa3ddc9a4a66b8dd7e8041bf0f22 P a78f88301ed54f48bf9e17f58fd62d5e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:39:42.677153 25171 tablet_replica.cc:330] T 461eaa3ddc9a4a66b8dd7e8041bf0f22 P 7c0a7ebfc3e74adeb520daaaf7609f27: stopping tablet replica
I20260812 02:39:42.677629 25171 raft_consensus.cc:2229] T 461eaa3ddc9a4a66b8dd7e8041bf0f22 P 7c0a7ebfc3e74adeb520daaaf7609f27 [term 1 LEADER]: Raft consensus shutting down.
I20260812 02:39:42.678413 25171 raft_consensus.cc:2258] T 461eaa3ddc9a4a66b8dd7e8041bf0f22 P 7c0a7ebfc3e74adeb520daaaf7609f27 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:39:42.679078 25172 ts_tablet_manager.cc:1905] T 461eaa3ddc9a4a66b8dd7e8041bf0f22 P a78f88301ed54f48bf9e17f58fd62d5e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:39:42.680495 25171 ts_tablet_manager.cc:1905] T 461eaa3ddc9a4a66b8dd7e8041bf0f22 P 7c0a7ebfc3e74adeb520daaaf7609f27: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 12 02:39:42 dist-test-slave-mn4r krb5kdc[24530](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502380, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.132.129@KRBTEST.COM
I20260812 02:39:42.687424 25172 ts_tablet_manager.cc:1918] T 461eaa3ddc9a4a66b8dd7e8041bf0f22 P a78f88301ed54f48bf9e17f58fd62d5e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 02:39:42.687704 25172 log.cc:1196] T 461eaa3ddc9a4a66b8dd7e8041bf0f22 P a78f88301ed54f48bf9e17f58fd62d5e: Deleting WAL directory at /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-0/wal/wals/461eaa3ddc9a4a66b8dd7e8041bf0f22
I20260812 02:39:42.688086 25171 ts_tablet_manager.cc:1918] T 461eaa3ddc9a4a66b8dd7e8041bf0f22 P 7c0a7ebfc3e74adeb520daaaf7609f27: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 02:39:42.688257 25171 log.cc:1196] T 461eaa3ddc9a4a66b8dd7e8041bf0f22 P 7c0a7ebfc3e74adeb520daaaf7609f27: Deleting WAL directory at /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-2/wal/wals/461eaa3ddc9a4a66b8dd7e8041bf0f22
I20260812 02:39:42.688500 25172 ts_tablet_manager.cc:1939] T 461eaa3ddc9a4a66b8dd7e8041bf0f22 P a78f88301ed54f48bf9e17f58fd62d5e: Deleting consensus metadata
I20260812 02:39:42.688838 25171 ts_tablet_manager.cc:1939] T 461eaa3ddc9a4a66b8dd7e8041bf0f22 P 7c0a7ebfc3e74adeb520daaaf7609f27: Deleting consensus metadata
I20260812 02:39:42.690670 24569 catalog_manager.cc:4841] TS 7c0a7ebfc3e74adeb520daaaf7609f27 (127.7.132.131:46073): tablet 461eaa3ddc9a4a66b8dd7e8041bf0f22 (table test-table [id=5646291aa1764707befe564d8567722b]) successfully deleted
I20260812 02:39:42.690980 24571 catalog_manager.cc:4841] TS a78f88301ed54f48bf9e17f58fd62d5e (127.7.132.129:41103): tablet 461eaa3ddc9a4a66b8dd7e8041bf0f22 (table test-table [id=5646291aa1764707befe564d8567722b]) successfully deleted
I20260812 02:39:42.695019 25139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:42.661429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40134 (local address 127.7.132.129:41103)
0812 02:39:42.661629 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0812 02:39:42.661635 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:39:42.674794 (+ 13159us) server_negotiation.cc:407] Connection header received
0812 02:39:42.675202 (+ 408us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:39:42.675212 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:39:42.675324 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:39:42.675422 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:39:42.678279 (+ 2857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:42.679413 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:42.681466 (+ 2053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:42.682157 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:42.682256 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:39:42.688174 (+ 5918us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:39:42.688189 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:39:42.688208 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:39:42.688257 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:39:42.692051 (+ 3794us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:39:42.692915 (+ 864us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:39:42.692922 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:39:42.692924 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:39:42.693094 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:39:42.693527 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:39:42.693530 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:39:42.693532 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:39:42.693862 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:39:42.694185 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0812 02:39:42.694517 (+ 332us) server_negotiation.cc:299] Negotiation successful
0812 02:39:42.694690 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260812 02:39:42.695739 24799 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:39:42.832552 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 24658
I20260812 02:39:42.864194 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 24814
Aug 12 02:39:42 dist-test-slave-mn4r krb5kdc[24530](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502380, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.132.130@KRBTEST.COM
I20260812 02:39:42.886859 25138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:42.865041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55642 (local address 127.7.132.130:41397)
0812 02:39:42.865247 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0812 02:39:42.865251 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 02:39:42.865306 (+ 55us) server_negotiation.cc:407] Connection header received
0812 02:39:42.865521 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:39:42.865525 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:39:42.865623 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:39:42.865702 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:39:42.867468 (+ 1766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:42.868253 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:42.870438 (+ 2185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:42.870846 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:42.870949 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:39:42.880458 (+ 9509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:39:42.880477 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:39:42.880494 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:39:42.880575 (+ 81us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:39:42.883720 (+ 3145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:39:42.884606 (+ 886us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:39:42.884614 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:39:42.884616 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:39:42.884756 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:39:42.885151 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:39:42.885155 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:39:42.885157 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:39:42.885524 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:39:42.885765 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0812 02:39:42.886211 (+ 446us) server_negotiation.cc:299] Negotiation successful
0812 02:39:42.886423 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260812 02:39:42.887631 24961 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:39:42.995358 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 24814
I20260812 02:39:43.025292 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 24976
Aug 12 02:39:43 dist-test-slave-mn4r krb5kdc[24530](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502380, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.132.131@KRBTEST.COM
I20260812 02:39:43.043713 25142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:43.026196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56778 (local address 127.7.132.131:46073)
0812 02:39:43.026371 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0812 02:39:43.026375 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 02:39:43.026454 (+ 79us) server_negotiation.cc:407] Connection header received
0812 02:39:43.026658 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:39:43.026662 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:39:43.026724 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:39:43.026824 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:39:43.028579 (+ 1755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:43.029363 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:43.031300 (+ 1937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:43.031764 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:43.031858 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:39:43.037448 (+ 5590us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:39:43.037460 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:39:43.037474 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:39:43.037508 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:39:43.040669 (+ 3161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:39:43.041392 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:39:43.041399 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:39:43.041402 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:39:43.041567 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:39:43.042009 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:39:43.042020 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:39:43.042022 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:39:43.042436 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:39:43.042817 (+ 381us) server_negotiation.cc:1035] Waiting for connection context
0812 02:39:43.043100 (+ 283us) server_negotiation.cc:299] Negotiation successful
0812 02:39:43.043340 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 02:39:43.044409 25114 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:39:43.168020 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 24976
I20260812 02:39:43.195389 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 24546
I20260812 02:39:43.196492 24623 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:39:43.308928 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 24546
I20260812 02:39:43.336158 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:44921
--webserver_interface=127.7.132.190
--webserver_port=45015
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:42213
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.132.190:44921
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:39:43.696573 25178 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:39:43.696874 25178 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:39:43.697057 25178 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:39:43.704418 25178 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:39:43.704527 25178 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:39:43.704629 25178 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:39:43.704728 25178 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:39:43.704779 25178 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:39:43.714944 25178 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:42213
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.7.132.190:44921
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:44921
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.132.190
--webserver_port=45015
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:39:43.716302 25178 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:39:43.717664 25178 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:39:43.743206 25185 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:39:43.743646 25188 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:39:43.743697 25186 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:39:43.744323 25178 server_base.cc:1034] running on GCE node
I20260812 02:39:43.745306 25178 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:39:43.747118 25178 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:39:43.748384 25178 hybrid_clock.cc:648] HybridClock initialized: now 1786502383748333 us; error 53 us; skew 500 ppm
I20260812 02:39:43.748816 25178 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:39:43 dist-test-slave-mn4r krb5kdc[24530](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502383, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:39:43.752599 25178 init.cc:377] Logged in from keytab as kudu/127.7.132.190@KRBTEST.COM (short username kudu)
I20260812 02:39:43.754381 25178 webserver.cc:460] Webserver started at http://127.7.132.190:45015/ using document root <none> and password file <none>
I20260812 02:39:43.755174 25178 fs_manager.cc:359] Metadata directory not provided
I20260812 02:39:43.755345 25178 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:39:43.761595 25178 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.002s
I20260812 02:39:43.764791 25196 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:39:43.766287 25178 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260812 02:39:43.766479 25178 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/master-0/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/master-0/wal
uuid: "ce6e17ddb54e4dc39dc6f8ef9c7f2d94"
format_stamp: "Formatted at 2026-08-12 02:39:40 on dist-test-slave-mn4r"
server_key: "9ba3905bf917b4c9aa9d00b47d3523df"
server_key_iv: "0338562476a92610dbf99c24a56e6b70"
server_key_version: "encryptionkey@0"
I20260812 02:39:43.767665 25178 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:39:43.792693 25178 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:39:43.797304 25178 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:39:43.797600 25178 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:39:43.817412 25178 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.190:44921
I20260812 02:39:43.817504 25256 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.190:44921 every 8 connection(s)
I20260812 02:39:43.819052 25178 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
I20260812 02:39:43.826198 25257 sys_catalog.cc:263] Verifying existing consensus state
I20260812 02:39:43.828068 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 25178
I20260812 02:39:43.828917 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.129:41103
--local_ip_for_outbound_sockets=127.7.132.129
--tserver_master_addrs=127.7.132.190:44921
--webserver_port=34017
--webserver_interface=127.7.132.129
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.129.keytab
--principal=kudu/127.7.132.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:42213
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 02:39:43.829649 25257 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce6e17ddb54e4dc39dc6f8ef9c7f2d94: Bootstrap starting.
I20260812 02:39:43.853085 25257 log.cc:824] T 00000000000000000000000000000000 P ce6e17ddb54e4dc39dc6f8ef9c7f2d94: Log is configured to *not* fsync() on all Append() calls
I20260812 02:39:43.870266 25257 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce6e17ddb54e4dc39dc6f8ef9c7f2d94: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 02:39:43.870980 25257 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce6e17ddb54e4dc39dc6f8ef9c7f2d94: Bootstrap complete.
I20260812 02:39:43.884866 25257 raft_consensus.cc:348] T 00000000000000000000000000000000 P ce6e17ddb54e4dc39dc6f8ef9c7f2d94 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce6e17ddb54e4dc39dc6f8ef9c7f2d94" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 44921 } }
I20260812 02:39:43.886111 25257 raft_consensus.cc:729] T 00000000000000000000000000000000 P ce6e17ddb54e4dc39dc6f8ef9c7f2d94 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce6e17ddb54e4dc39dc6f8ef9c7f2d94, State: Initialized, Role: FOLLOWER
I20260812 02:39:43.886847 25257 consensus_queue.cc:260] T 00000000000000000000000000000000 P ce6e17ddb54e4dc39dc6f8ef9c7f2d94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce6e17ddb54e4dc39dc6f8ef9c7f2d94" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 44921 } }
I20260812 02:39:43.887135 25257 raft_consensus.cc:388] T 00000000000000000000000000000000 P ce6e17ddb54e4dc39dc6f8ef9c7f2d94 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 02:39:43.887259 25257 raft_consensus.cc:482] T 00000000000000000000000000000000 P ce6e17ddb54e4dc39dc6f8ef9c7f2d94 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 02:39:43.887439 25257 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ce6e17ddb54e4dc39dc6f8ef9c7f2d94 [term 1 FOLLOWER]: Advancing to term 2
I20260812 02:39:43.893095 25257 raft_consensus.cc:504] T 00000000000000000000000000000000 P ce6e17ddb54e4dc39dc6f8ef9c7f2d94 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce6e17ddb54e4dc39dc6f8ef9c7f2d94" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 44921 } }
I20260812 02:39:43.893747 25257 leader_election.cc:302] T 00000000000000000000000000000000 P ce6e17ddb54e4dc39dc6f8ef9c7f2d94 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ce6e17ddb54e4dc39dc6f8ef9c7f2d94; no voters:
I20260812 02:39:43.894459 25257 leader_election.cc:288] T 00000000000000000000000000000000 P ce6e17ddb54e4dc39dc6f8ef9c7f2d94 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 02:39:43.894661 25262 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ce6e17ddb54e4dc39dc6f8ef9c7f2d94 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 02:39:43.896016 25262 raft_consensus.cc:686] T 00000000000000000000000000000000 P ce6e17ddb54e4dc39dc6f8ef9c7f2d94 [term 2 LEADER]: Becoming Leader. State: Replica: ce6e17ddb54e4dc39dc6f8ef9c7f2d94, State: Running, Role: LEADER
I20260812 02:39:43.896682 25262 consensus_queue.cc:237] T 00000000000000000000000000000000 P ce6e17ddb54e4dc39dc6f8ef9c7f2d94 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce6e17ddb54e4dc39dc6f8ef9c7f2d94" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 44921 } }
I20260812 02:39:43.896965 25257 sys_catalog.cc:564] T 00000000000000000000000000000000 P ce6e17ddb54e4dc39dc6f8ef9c7f2d94 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:39:43.899358 25264 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce6e17ddb54e4dc39dc6f8ef9c7f2d94 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ce6e17ddb54e4dc39dc6f8ef9c7f2d94. Latest consensus state: current_term: 2 leader_uuid: "ce6e17ddb54e4dc39dc6f8ef9c7f2d94" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce6e17ddb54e4dc39dc6f8ef9c7f2d94" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 44921 } } }
I20260812 02:39:43.899822 25264 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce6e17ddb54e4dc39dc6f8ef9c7f2d94 [sys.catalog]: This master's current role is: LEADER
I20260812 02:39:43.900501 25263 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce6e17ddb54e4dc39dc6f8ef9c7f2d94 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "ce6e17ddb54e4dc39dc6f8ef9c7f2d94" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce6e17ddb54e4dc39dc6f8ef9c7f2d94" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 44921 } } }
I20260812 02:39:43.900897 25263 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce6e17ddb54e4dc39dc6f8ef9c7f2d94 [sys.catalog]: This master's current role is: LEADER
I20260812 02:39:43.922359 25269 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:39:43.932893 25269 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 02:39:43.934609 25269 catalog_manager.cc:1259] Loaded cluster ID: a19eed9dfb2546c89c27bf5be3380eb7
I20260812 02:39:43.934798 25269 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 02:39:43.938458 25269 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260812 02:39:43.959074 25269 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260812 02:39:43.971154 25282 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260812 02:39:43.975637 25269 catalog_manager.cc:1504] Loading token signing keys...
I20260812 02:39:43.977408 25269 catalog_manager.cc:5879] T 00000000000000000000000000000000 P ce6e17ddb54e4dc39dc6f8ef9c7f2d94: Loaded TSK: 0
I20260812 02:39:43.979061 25269 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 02:39:44.229691 25259 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:39:44.229967 25259 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:39:44.230150 25259 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:39:44.237520 25259 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:39:44.237691 25259 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:39:44.237887 25259 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.129
I20260812 02:39:44.247668 25259 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:42213
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.129.keytab
--principal=kudu/127.7.132.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.129:41103
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.132.129
--webserver_port=34017
--tserver_master_addrs=127.7.132.190:44921
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.129
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:39:44.249040 25259 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:39:44.250492 25259 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:39:44.264108 25290 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:39:44.264387 25289 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:39:44.264815 25292 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:39:44.265964 25259 server_base.cc:1034] running on GCE node
I20260812 02:39:44.266759 25259 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:39:44.267805 25259 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:39:44.269032 25259 hybrid_clock.cc:648] HybridClock initialized: now 1786502384268965 us; error 57 us; skew 500 ppm
I20260812 02:39:44.269366 25259 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:39:44 dist-test-slave-mn4r krb5kdc[24530](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502384, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:39:44.272719 25259 init.cc:377] Logged in from keytab as kudu/127.7.132.129@KRBTEST.COM (short username kudu)
I20260812 02:39:44.274266 25259 webserver.cc:460] Webserver started at http://127.7.132.129:34017/ using document root <none> and password file <none>
I20260812 02:39:44.274973 25259 fs_manager.cc:359] Metadata directory not provided
I20260812 02:39:44.275130 25259 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:39:44.280599 25259 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.000s sys 0.005s
I20260812 02:39:44.283595 25301 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:39:44.285091 25259 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260812 02:39:44.285279 25259 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-0/wal
uuid: "a78f88301ed54f48bf9e17f58fd62d5e"
format_stamp: "Formatted at 2026-08-12 02:39:41 on dist-test-slave-mn4r"
server_key: "7b5f87b6450b31d2c4b0e385768e0548"
server_key_iv: "90cbe381b2dc9e8fe4cac009c9028cfa"
server_key_version: "encryptionkey@0"
I20260812 02:39:44.286317 25259 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:39:44.306756 25259 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:39:44.311407 25259 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:39:44.311717 25259 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:39:44.312956 25259 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:39:44.314536 25259 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:39:44.314661 25259 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:39:44.314803 25259 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:39:44.314893 25259 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:39:44.356519 25259 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.129:41103
I20260812 02:39:44.356617 25427 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.129:41103 every 8 connection(s)
I20260812 02:39:44.358204 25259 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-0/data/info.pb
I20260812 02:39:44.363857 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 25259
I20260812 02:39:44.364516 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.130:41397
--local_ip_for_outbound_sockets=127.7.132.130
--tserver_master_addrs=127.7.132.190:44921
--webserver_port=36867
--webserver_interface=127.7.132.130
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.130.keytab
--principal=kudu/127.7.132.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:42213
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 02:39:44 dist-test-slave-mn4r krb5kdc[24530](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502384, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.129@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 02:39:44.383081 25430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:44.361127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:34547 (local address 127.7.132.190:44921)
0812 02:39:44.361694 (+ 567us) server_negotiation.cc:206] Beginning negotiation
0812 02:39:44.361708 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 02:39:44.362516 (+ 808us) server_negotiation.cc:407] Connection header received
0812 02:39:44.364906 (+ 2390us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:39:44.364919 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:39:44.365291 (+ 372us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:39:44.365677 (+ 386us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:39:44.367341 (+ 1664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:44.368601 (+ 1260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:44.370240 (+ 1639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:44.370888 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:44.371099 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:39:44.375111 (+ 4012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:39:44.375125 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:39:44.375147 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:39:44.375206 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:39:44.378351 (+ 3145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:39:44.379347 (+ 996us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:39:44.379355 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:39:44.379361 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:39:44.379546 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:39:44.380055 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:39:44.380058 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:39:44.380060 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:39:44.380546 (+ 486us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:39:44.380700 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0812 02:39:44.381705 (+ 1005us) server_negotiation.cc:299] Negotiation successful
0812 02:39:44.382049 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":238,"threads_started":1}
I20260812 02:39:44.383596 25431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:44.361455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:44921 (local address 127.7.132.129:34547)
0812 02:39:44.362314 (+ 859us) negotiation.cc:107] Waiting for socket to connect
0812 02:39:44.362355 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 02:39:44.364231 (+ 1876us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:39:44.365843 (+ 1612us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:39:44.365854 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:39:44.366523 (+ 669us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:39:44.367087 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:39:44.367106 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:39:44.368785 (+ 1679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:39:44.368789 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:39:44.370029 (+ 1240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:39:44.370043 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:39:44.371143 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:39:44.371152 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:39:44.371481 (+ 329us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:39:44.372314 (+ 833us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:39:44.372337 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:39:44.374782 (+ 2445us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:39:44.378509 (+ 3727us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:39:44.378521 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:39:44.378540 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:39:44.379190 (+ 650us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:39:44.379665 (+ 475us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:39:44.379673 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:39:44.379679 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:39:44.379932 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:39:44.380702 (+ 770us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:39:44.380713 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:39:44.381152 (+ 439us) client_negotiation.cc:769] Sending connection context
0812 02:39:44.381471 (+ 319us) client_negotiation.cc:240] Negotiation successful
0812 02:39:44.382176 (+ 705us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":561,"thread_start_us":306,"threads_started":1}
I20260812 02:39:44.386538 25428 heartbeater.cc:347] Connected to a master server at 127.7.132.190:44921
I20260812 02:39:44.386866 25428 heartbeater.cc:464] Registering TS with master...
I20260812 02:39:44.387619 25428 heartbeater.cc:511] Master 127.7.132.190:44921 requested a full tablet report, sending...
I20260812 02:39:44.389932 25216 ts_manager.cc:194] Registered new tserver with Master: a78f88301ed54f48bf9e17f58fd62d5e (127.7.132.129:41103)
I20260812 02:39:44.391728 25216 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.129@KRBTEST.COM'} at 127.7.132.129:34547
W20260812 02:39:44.713868 25432 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:39:44.714267 25432 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:39:44.714382 25432 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:39:44.721504 25432 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:39:44.721606 25432 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:39:44.721716 25432 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.130
I20260812 02:39:44.731524 25432 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:42213
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.130.keytab
--principal=kudu/127.7.132.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.130:41397
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.132.130
--webserver_port=36867
--tserver_master_addrs=127.7.132.190:44921
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.130
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:39:44.732909 25432 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:39:44.734290 25432 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:39:44.747008 25440 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:39:44.747125 25441 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:39:44.747156 25443 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:39:44.748090 25432 server_base.cc:1034] running on GCE node
I20260812 02:39:44.748761 25432 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:39:44.749711 25432 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:39:44.750929 25432 hybrid_clock.cc:648] HybridClock initialized: now 1786502384750875 us; error 55 us; skew 500 ppm
I20260812 02:39:44.751278 25432 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:39:44 dist-test-slave-mn4r krb5kdc[24530](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502384, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:39:44.754698 25432 init.cc:377] Logged in from keytab as kudu/127.7.132.130@KRBTEST.COM (short username kudu)
I20260812 02:39:44.756362 25432 webserver.cc:460] Webserver started at http://127.7.132.130:36867/ using document root <none> and password file <none>
I20260812 02:39:44.757169 25432 fs_manager.cc:359] Metadata directory not provided
I20260812 02:39:44.757351 25432 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:39:44.763410 25432 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 02:39:44.766534 25452 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:39:44.768133 25432 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260812 02:39:44.768344 25432 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-1/wal
uuid: "9a848619566249798e7b9f2d4118c664"
format_stamp: "Formatted at 2026-08-12 02:39:41 on dist-test-slave-mn4r"
server_key: "8d3147d5c2e5775b5e5295a7d3f1933e"
server_key_iv: "caf67665118634047b0df87a316228e8"
server_key_version: "encryptionkey@0"
I20260812 02:39:44.769428 25432 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:39:44.785203 25432 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:39:44.790277 25432 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:39:44.790607 25432 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:39:44.791697 25432 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:39:44.793215 25432 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:39:44.793341 25432 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:39:44.793500 25432 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:39:44.793586 25432 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:39:44.831401 25432 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.130:41397
I20260812 02:39:44.831447 25577 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.130:41397 every 8 connection(s)
I20260812 02:39:44.833209 25432 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-1/data/info.pb
I20260812 02:39:44.835891 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 25432
I20260812 02:39:44.836603 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.131:46073
--local_ip_for_outbound_sockets=127.7.132.131
--tserver_master_addrs=127.7.132.190:44921
--webserver_port=41163
--webserver_interface=127.7.132.131
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.131.keytab
--principal=kudu/127.7.132.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:42213
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 02:39:44 dist-test-slave-mn4r krb5kdc[24530](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502384, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.130@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 02:39:44.855517 25430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:44.836097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:54043 (local address 127.7.132.190:44921)
0812 02:39:44.836324 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0812 02:39:44.836330 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:39:44.837808 (+ 1478us) server_negotiation.cc:407] Connection header received
0812 02:39:44.839998 (+ 2190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:39:44.840005 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:39:44.840122 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:39:44.840248 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:39:44.841930 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:44.842801 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:44.844550 (+ 1749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:44.845027 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:44.845187 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:39:44.849013 (+ 3826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:39:44.849019 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:39:44.849022 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:39:44.849067 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:39:44.851971 (+ 2904us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:39:44.852809 (+ 838us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:39:44.852817 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:39:44.852821 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:39:44.852989 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:39:44.853487 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:39:44.853493 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:39:44.853497 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:39:44.853796 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:39:44.853997 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0812 02:39:44.854848 (+ 851us) server_negotiation.cc:299] Negotiation successful
0812 02:39:44.855130 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260812 02:39:44.856650 25580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:44.836706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:44921 (local address 127.7.132.130:54043)
0812 02:39:44.837617 (+ 911us) negotiation.cc:107] Waiting for socket to connect
0812 02:39:44.837675 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0812 02:39:44.839683 (+ 2008us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:39:44.840535 (+ 852us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:39:44.840544 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:39:44.841088 (+ 544us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:39:44.841703 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:39:44.841723 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:39:44.842992 (+ 1269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:39:44.842998 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:39:44.844362 (+ 1364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:39:44.844376 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:39:44.845288 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:39:44.845296 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:39:44.845556 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:39:44.846366 (+ 810us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:39:44.846406 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:39:44.848767 (+ 2361us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:39:44.852153 (+ 3386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:39:44.852163 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:39:44.852180 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:39:44.852668 (+ 488us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:39:44.853120 (+ 452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:39:44.853125 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:39:44.853129 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:39:44.853343 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:39:44.853968 (+ 625us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:39:44.853977 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:39:44.854434 (+ 457us) client_negotiation.cc:769] Sending connection context
0812 02:39:44.854745 (+ 311us) client_negotiation.cc:240] Negotiation successful
0812 02:39:44.855341 (+ 596us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":598,"thread_start_us":391,"threads_started":1}
I20260812 02:39:44.858561 25578 heartbeater.cc:347] Connected to a master server at 127.7.132.190:44921
I20260812 02:39:44.858935 25578 heartbeater.cc:464] Registering TS with master...
I20260812 02:39:44.859722 25578 heartbeater.cc:511] Master 127.7.132.190:44921 requested a full tablet report, sending...
I20260812 02:39:44.861217 25216 ts_manager.cc:194] Registered new tserver with Master: 9a848619566249798e7b9f2d4118c664 (127.7.132.130:41397)
I20260812 02:39:44.862068 25216 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.130@KRBTEST.COM'} at 127.7.132.130:54043
W20260812 02:39:45.179173 25581 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:39:45.179551 25581 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:39:45.179735 25581 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:39:45.187068 25581 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:39:45.187222 25581 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:39:45.187418 25581 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.131
I20260812 02:39:45.198956 25581 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:42213
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.131.keytab
--principal=kudu/127.7.132.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.131:46073
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.132.131
--webserver_port=41163
--tserver_master_addrs=127.7.132.190:44921
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.131
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:39:45.200379 25581 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:39:45.201812 25581 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:39:45.215236 25589 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:39:45.215327 25592 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:39:45.215229 25590 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:39:45.216295 25581 server_base.cc:1034] running on GCE node
I20260812 02:39:45.216985 25581 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:39:45.217932 25581 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:39:45.219156 25581 hybrid_clock.cc:648] HybridClock initialized: now 1786502385219110 us; error 61 us; skew 500 ppm
I20260812 02:39:45.219470 25581 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:39:45 dist-test-slave-mn4r krb5kdc[24530](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502385, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:39:45.222759 25581 init.cc:377] Logged in from keytab as kudu/127.7.132.131@KRBTEST.COM (short username kudu)
I20260812 02:39:45.224265 25581 webserver.cc:460] Webserver started at http://127.7.132.131:41163/ using document root <none> and password file <none>
I20260812 02:39:45.224956 25581 fs_manager.cc:359] Metadata directory not provided
I20260812 02:39:45.225117 25581 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:39:45.230595 25581 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.002s
I20260812 02:39:45.233381 25599 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:39:45.234764 25581 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 02:39:45.234938 25581 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-2/wal
uuid: "7c0a7ebfc3e74adeb520daaaf7609f27"
format_stamp: "Formatted at 2026-08-12 02:39:42 on dist-test-slave-mn4r"
server_key: "82d319a99df0be3d1797d77d9546b6aa"
server_key_iv: "763557f03a3893646fb09d441e17cf97"
server_key_version: "encryptionkey@0"
I20260812 02:39:45.235913 25581 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:39:45.260624 25581 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:39:45.264699 25581 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:39:45.264976 25581 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:39:45.266053 25581 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:39:45.267429 25581 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:39:45.267553 25581 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:39:45.267686 25581 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:39:45.267773 25581 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:39:45.303079 25581 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.131:46073
I20260812 02:39:45.303200 25734 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.131:46073 every 8 connection(s)
I20260812 02:39:45.304601 25581 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-2/data/info.pb
I20260812 02:39:45.306382 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 25581
Aug 12 02:39:45 dist-test-slave-mn4r krb5kdc[24530](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502385, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.131@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 02:39:45.322954 25430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:45.307474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42724 (local address 127.7.132.190:44921)
0812 02:39:45.307665 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0812 02:39:45.307674 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 02:39:45.308247 (+ 573us) server_negotiation.cc:407] Connection header received
0812 02:39:45.308431 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:39:45.308435 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:39:45.308534 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:39:45.308663 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:39:45.310455 (+ 1792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:45.311653 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:45.313048 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:45.313606 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:45.313700 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:39:45.316940 (+ 3240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:39:45.316947 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:39:45.316954 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:39:45.317039 (+ 85us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:39:45.320186 (+ 3147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:39:45.320835 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:39:45.320843 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:39:45.320845 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:39:45.320967 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:39:45.321381 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:39:45.321384 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:39:45.321386 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:39:45.321650 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:39:45.321958 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0812 02:39:45.322346 (+ 388us) server_negotiation.cc:299] Negotiation successful
0812 02:39:45.322587 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":47}
I20260812 02:39:45.327427 25738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:45.307474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:41781 (local address 127.7.132.190:44921)
0812 02:39:45.308335 (+ 861us) server_negotiation.cc:206] Beginning negotiation
0812 02:39:45.308339 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 02:39:45.309208 (+ 869us) server_negotiation.cc:407] Connection header received
0812 02:39:45.311461 (+ 2253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:39:45.311465 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:39:45.311527 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:39:45.311615 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:39:45.313186 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:45.314184 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:45.315841 (+ 1657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:45.316584 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:45.316714 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:39:45.320859 (+ 4145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:39:45.320864 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:39:45.320868 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:39:45.320908 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:39:45.324105 (+ 3197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:39:45.324933 (+ 828us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:39:45.324937 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:39:45.324939 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:39:45.325041 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:39:45.325539 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:39:45.325546 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:39:45.325549 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:39:45.325912 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:39:45.326071 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0812 02:39:45.326785 (+ 714us) server_negotiation.cc:299] Negotiation successful
0812 02:39:45.327001 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":217,"server-negotiator.queue_time_us":760,"thread_start_us":434,"threads_started":1}
I20260812 02:39:45.328383 25739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:45.308146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:44921 (local address 127.7.132.131:41781)
0812 02:39:45.309014 (+ 868us) negotiation.cc:107] Waiting for socket to connect
0812 02:39:45.309094 (+ 80us) client_negotiation.cc:174] Beginning negotiation
0812 02:39:45.311171 (+ 2077us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:39:45.311902 (+ 731us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:39:45.311912 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:39:45.312408 (+ 496us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:39:45.312989 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:39:45.313016 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:39:45.314383 (+ 1367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:39:45.314388 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:39:45.315685 (+ 1297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:39:45.315697 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:39:45.316798 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:39:45.316828 (+ 30us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:39:45.317159 (+ 331us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:39:45.317827 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:39:45.317861 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:39:45.320630 (+ 2769us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:39:45.324237 (+ 3607us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:39:45.324245 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:39:45.324266 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:39:45.324827 (+ 561us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:39:45.325161 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:39:45.325165 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:39:45.325168 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:39:45.325409 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:39:45.326049 (+ 640us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:39:45.326057 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:39:45.326426 (+ 369us) client_negotiation.cc:769] Sending connection context
0812 02:39:45.326706 (+ 280us) client_negotiation.cc:240] Negotiation successful
0812 02:39:45.327259 (+ 553us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":578,"thread_start_us":359,"threads_started":1}
I20260812 02:39:45.329772 25735 heartbeater.cc:347] Connected to a master server at 127.7.132.190:44921
I20260812 02:39:45.330058 25735 heartbeater.cc:464] Registering TS with master...
I20260812 02:39:45.330824 25735 heartbeater.cc:511] Master 127.7.132.190:44921 requested a full tablet report, sending...
I20260812 02:39:45.332365 25216 ts_manager.cc:194] Registered new tserver with Master: 7c0a7ebfc3e74adeb520daaaf7609f27 (127.7.132.131:46073)
I20260812 02:39:45.333196 25216 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.131@KRBTEST.COM'} at 127.7.132.131:41781
I20260812 02:39:45.338056 7698 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 02:39:45.345868 25738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:45.338990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42736 (local address 127.7.132.190:44921)
0812 02:39:45.339202 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0812 02:39:45.339207 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:39:45.339739 (+ 532us) server_negotiation.cc:407] Connection header received
0812 02:39:45.339940 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:39:45.339945 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:39:45.340124 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:39:45.340294 (+ 170us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 02:39:45.340732 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:45.341437 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:45.342899 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:45.343346 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:45.343440 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:39:45.344141 (+ 701us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 02:39:45.345078 (+ 937us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 02:39:45.345277 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0812 02:39:45.345414 (+ 137us) server_negotiation.cc:299] Negotiation successful
0812 02:39:45.345539 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260812 02:39:45.347131 25216 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:42736:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 02:39:45.357086 25216 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 02:39:45.395058 25428 heartbeater.cc:503] Master 127.7.132.190:44921 was elected leader, sending a full tablet report...
I20260812 02:39:45.411888 25745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:45.370144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:41103 (local address 127.0.0.1:40142)
0812 02:39:45.386874 (+ 16730us) negotiation.cc:107] Waiting for socket to connect
0812 02:39:45.386939 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0812 02:39:45.387197 (+ 258us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:39:45.401276 (+ 14079us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:39:45.401284 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:39:45.401332 (+ 48us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:39:45.403702 (+ 2370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:39:45.403724 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:39:45.405740 (+ 2016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:39:45.405745 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:39:45.408499 (+ 2754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:39:45.408512 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:39:45.410803 (+ 2291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:39:45.410809 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:39:45.411047 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:39:45.411114 (+ 67us) client_negotiation.cc:769] Sending connection context
0812 02:39:45.411233 (+ 119us) client_negotiation.cc:240] Negotiation successful
0812 02:39:45.411433 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15767,"mutex_wait_us":1,"thread_start_us":15562,"threads_started":1}
I20260812 02:39:45.412310 25750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:45.370606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56790 (local address 127.7.132.131:46073)
0812 02:39:45.402753 (+ 32147us) server_negotiation.cc:206] Beginning negotiation
0812 02:39:45.402761 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:39:45.403411 (+ 650us) server_negotiation.cc:407] Connection header received
0812 02:39:45.404155 (+ 744us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:39:45.404163 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:39:45.404382 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:39:45.404588 (+ 206us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:39:45.405212 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:45.406665 (+ 1453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:45.409476 (+ 2811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:45.410971 (+ 1495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:45.411089 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:39:45.411309 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0812 02:39:45.411789 (+ 480us) server_negotiation.cc:299] Negotiation successful
0812 02:39:45.411969 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31963,"thread_start_us":31789,"threads_started":1}
I20260812 02:39:45.412688 25747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:45.370139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.131:46073 (local address 127.0.0.1:56790)
0812 02:39:45.403206 (+ 33067us) negotiation.cc:107] Waiting for socket to connect
0812 02:39:45.403226 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 02:39:45.403804 (+ 578us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:39:45.404572 (+ 768us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:39:45.404576 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:39:45.404602 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:39:45.404996 (+ 394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:39:45.405013 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:39:45.406842 (+ 1829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:39:45.406873 (+ 31us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:39:45.409257 (+ 2384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:39:45.409273 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:39:45.411449 (+ 2176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:39:45.411454 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:39:45.411595 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:39:45.411617 (+ 22us) client_negotiation.cc:769] Sending connection context
0812 02:39:45.411729 (+ 112us) client_negotiation.cc:240] Negotiation successful
0812 02:39:45.411868 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32924,"mutex_wait_us":56,"thread_start_us":32663,"threads_started":1}
I20260812 02:39:45.413255 25751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:45.370765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.130:41397 (local address 127.0.0.1:55650)
0812 02:39:45.403891 (+ 33126us) negotiation.cc:107] Waiting for socket to connect
0812 02:39:45.403917 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0812 02:39:45.404039 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:39:45.404898 (+ 859us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:39:45.404904 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:39:45.404936 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:39:45.405319 (+ 383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:39:45.405327 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:39:45.407653 (+ 2326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:39:45.407660 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:39:45.409839 (+ 2179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:39:45.409854 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:39:45.412363 (+ 2509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:39:45.412404 (+ 41us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:39:45.412614 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:39:45.412657 (+ 43us) client_negotiation.cc:769] Sending connection context
0812 02:39:45.412737 (+ 80us) client_negotiation.cc:240] Negotiation successful
0812 02:39:45.412891 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32976,"mutex_wait_us":1,"thread_start_us":31043,"threads_started":1}
I20260812 02:39:45.413344 25753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:45.371139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55650 (local address 127.7.132.130:41397)
0812 02:39:45.399417 (+ 28278us) server_negotiation.cc:206] Beginning negotiation
0812 02:39:45.399432 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 02:39:45.404175 (+ 4743us) server_negotiation.cc:407] Connection header received
0812 02:39:45.404291 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:39:45.404300 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:39:45.404577 (+ 277us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:39:45.404771 (+ 194us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:39:45.405513 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:45.407490 (+ 1977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:45.410030 (+ 2540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:45.411847 (+ 1817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:45.411994 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:39:45.412208 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0812 02:39:45.412831 (+ 623us) server_negotiation.cc:299] Negotiation successful
0812 02:39:45.413040 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28007,"thread_start_us":28306,"threads_started":1}
I20260812 02:39:45.413805 25749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:45.370586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40142 (local address 127.7.132.129:41103)
0812 02:39:45.400260 (+ 29674us) server_negotiation.cc:206] Beginning negotiation
0812 02:39:45.400273 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 02:39:45.400377 (+ 104us) server_negotiation.cc:407] Connection header received
0812 02:39:45.400587 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:39:45.400602 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:39:45.400981 (+ 379us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:39:45.401261 (+ 280us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:39:45.403980 (+ 2719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:45.405586 (+ 1606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:45.409248 (+ 3662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:45.410555 (+ 1307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:45.411985 (+ 1430us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:39:45.413203 (+ 1218us) server_negotiation.cc:1035] Waiting for connection context
0812 02:39:45.413281 (+ 78us) server_negotiation.cc:299] Negotiation successful
0812 02:39:45.413473 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29454,"thread_start_us":31190,"threads_started":1}
I20260812 02:39:45.416548 25656 tablet_service.cc:1467] Processing CreateTablet for tablet 5963125c6453469db15e9c9b183b1856 (DEFAULT_TABLE table=test-table [id=13107de8e5a048c08606d96655df38ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:39:45.416997 25506 tablet_service.cc:1467] Processing CreateTablet for tablet 5963125c6453469db15e9c9b183b1856 (DEFAULT_TABLE table=test-table [id=13107de8e5a048c08606d96655df38ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:39:45.417868 25656 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5963125c6453469db15e9c9b183b1856. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:39:45.418277 25506 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5963125c6453469db15e9c9b183b1856. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:39:45.425388 25354 tablet_service.cc:1467] Processing CreateTablet for tablet 5963125c6453469db15e9c9b183b1856 (DEFAULT_TABLE table=test-table [id=13107de8e5a048c08606d96655df38ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:39:45.427006 25354 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5963125c6453469db15e9c9b183b1856. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:39:45.431766 25754 tablet_bootstrap.cc:492] T 5963125c6453469db15e9c9b183b1856 P 9a848619566249798e7b9f2d4118c664: Bootstrap starting.
I20260812 02:39:45.433713 25755 tablet_bootstrap.cc:492] T 5963125c6453469db15e9c9b183b1856 P 7c0a7ebfc3e74adeb520daaaf7609f27: Bootstrap starting.
I20260812 02:39:45.436020 25754 tablet_bootstrap.cc:654] T 5963125c6453469db15e9c9b183b1856 P 9a848619566249798e7b9f2d4118c664: Neither blocks nor log segments found. Creating new log.
I20260812 02:39:45.437456 25755 tablet_bootstrap.cc:654] T 5963125c6453469db15e9c9b183b1856 P 7c0a7ebfc3e74adeb520daaaf7609f27: Neither blocks nor log segments found. Creating new log.
I20260812 02:39:45.437564 25754 log.cc:824] T 5963125c6453469db15e9c9b183b1856 P 9a848619566249798e7b9f2d4118c664: Log is configured to *not* fsync() on all Append() calls
I20260812 02:39:45.438761 25755 log.cc:824] T 5963125c6453469db15e9c9b183b1856 P 7c0a7ebfc3e74adeb520daaaf7609f27: Log is configured to *not* fsync() on all Append() calls
I20260812 02:39:45.441031 25754 tablet_bootstrap.cc:492] T 5963125c6453469db15e9c9b183b1856 P 9a848619566249798e7b9f2d4118c664: No bootstrap required, opened a new log
I20260812 02:39:45.441337 25754 ts_tablet_manager.cc:1397] T 5963125c6453469db15e9c9b183b1856 P 9a848619566249798e7b9f2d4118c664: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.002s
I20260812 02:39:45.442039 25755 tablet_bootstrap.cc:492] T 5963125c6453469db15e9c9b183b1856 P 7c0a7ebfc3e74adeb520daaaf7609f27: No bootstrap required, opened a new log
I20260812 02:39:45.442361 25755 ts_tablet_manager.cc:1397] T 5963125c6453469db15e9c9b183b1856 P 7c0a7ebfc3e74adeb520daaaf7609f27: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.005s
I20260812 02:39:45.442703 25759 tablet_bootstrap.cc:492] T 5963125c6453469db15e9c9b183b1856 P a78f88301ed54f48bf9e17f58fd62d5e: Bootstrap starting.
I20260812 02:39:45.446071 25759 tablet_bootstrap.cc:654] T 5963125c6453469db15e9c9b183b1856 P a78f88301ed54f48bf9e17f58fd62d5e: Neither blocks nor log segments found. Creating new log.
I20260812 02:39:45.447077 25759 log.cc:824] T 5963125c6453469db15e9c9b183b1856 P a78f88301ed54f48bf9e17f58fd62d5e: Log is configured to *not* fsync() on all Append() calls
I20260812 02:39:45.449561 25759 tablet_bootstrap.cc:492] T 5963125c6453469db15e9c9b183b1856 P a78f88301ed54f48bf9e17f58fd62d5e: No bootstrap required, opened a new log
I20260812 02:39:45.449824 25759 ts_tablet_manager.cc:1397] T 5963125c6453469db15e9c9b183b1856 P a78f88301ed54f48bf9e17f58fd62d5e: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260812 02:39:45.452497 25754 raft_consensus.cc:348] T 5963125c6453469db15e9c9b183b1856 P 9a848619566249798e7b9f2d4118c664 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0a7ebfc3e74adeb520daaaf7609f27" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 46073 } } peers { permanent_uuid: "a78f88301ed54f48bf9e17f58fd62d5e" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 41103 } } peers { permanent_uuid: "9a848619566249798e7b9f2d4118c664" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 41397 } }
I20260812 02:39:45.452997 25754 raft_consensus.cc:374] T 5963125c6453469db15e9c9b183b1856 P 9a848619566249798e7b9f2d4118c664 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:39:45.453161 25754 raft_consensus.cc:729] T 5963125c6453469db15e9c9b183b1856 P 9a848619566249798e7b9f2d4118c664 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a848619566249798e7b9f2d4118c664, State: Initialized, Role: FOLLOWER
I20260812 02:39:45.453100 25755 raft_consensus.cc:348] T 5963125c6453469db15e9c9b183b1856 P 7c0a7ebfc3e74adeb520daaaf7609f27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0a7ebfc3e74adeb520daaaf7609f27" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 46073 } } peers { permanent_uuid: "a78f88301ed54f48bf9e17f58fd62d5e" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 41103 } } peers { permanent_uuid: "9a848619566249798e7b9f2d4118c664" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 41397 } }
I20260812 02:39:45.453593 25755 raft_consensus.cc:374] T 5963125c6453469db15e9c9b183b1856 P 7c0a7ebfc3e74adeb520daaaf7609f27 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:39:45.453704 25755 raft_consensus.cc:729] T 5963125c6453469db15e9c9b183b1856 P 7c0a7ebfc3e74adeb520daaaf7609f27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c0a7ebfc3e74adeb520daaaf7609f27, State: Initialized, Role: FOLLOWER
I20260812 02:39:45.453850 25754 consensus_queue.cc:260] T 5963125c6453469db15e9c9b183b1856 P 9a848619566249798e7b9f2d4118c664 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0a7ebfc3e74adeb520daaaf7609f27" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 46073 } } peers { permanent_uuid: "a78f88301ed54f48bf9e17f58fd62d5e" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 41103 } } peers { permanent_uuid: "9a848619566249798e7b9f2d4118c664" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 41397 } }
I20260812 02:39:45.454332 25755 consensus_queue.cc:260] T 5963125c6453469db15e9c9b183b1856 P 7c0a7ebfc3e74adeb520daaaf7609f27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0a7ebfc3e74adeb520daaaf7609f27" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 46073 } } peers { permanent_uuid: "a78f88301ed54f48bf9e17f58fd62d5e" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 41103 } } peers { permanent_uuid: "9a848619566249798e7b9f2d4118c664" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 41397 } }
I20260812 02:39:45.455227 25578 heartbeater.cc:503] Master 127.7.132.190:44921 was elected leader, sending a full tablet report...
I20260812 02:39:45.456223 25754 ts_tablet_manager.cc:1428] T 5963125c6453469db15e9c9b183b1856 P 9a848619566249798e7b9f2d4118c664: Time spent starting tablet: real 0.015s user 0.010s sys 0.005s
I20260812 02:39:45.458880 25735 heartbeater.cc:503] Master 127.7.132.190:44921 was elected leader, sending a full tablet report...
I20260812 02:39:45.459501 25755 ts_tablet_manager.cc:1428] T 5963125c6453469db15e9c9b183b1856 P 7c0a7ebfc3e74adeb520daaaf7609f27: Time spent starting tablet: real 0.017s user 0.011s sys 0.005s
I20260812 02:39:45.462255 25759 raft_consensus.cc:348] T 5963125c6453469db15e9c9b183b1856 P a78f88301ed54f48bf9e17f58fd62d5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0a7ebfc3e74adeb520daaaf7609f27" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 46073 } } peers { permanent_uuid: "a78f88301ed54f48bf9e17f58fd62d5e" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 41103 } } peers { permanent_uuid: "9a848619566249798e7b9f2d4118c664" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 41397 } }
I20260812 02:39:45.462738 25759 raft_consensus.cc:374] T 5963125c6453469db15e9c9b183b1856 P a78f88301ed54f48bf9e17f58fd62d5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:39:45.462854 25759 raft_consensus.cc:729] T 5963125c6453469db15e9c9b183b1856 P a78f88301ed54f48bf9e17f58fd62d5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a78f88301ed54f48bf9e17f58fd62d5e, State: Initialized, Role: FOLLOWER
I20260812 02:39:45.463485 25759 consensus_queue.cc:260] T 5963125c6453469db15e9c9b183b1856 P a78f88301ed54f48bf9e17f58fd62d5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0a7ebfc3e74adeb520daaaf7609f27" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 46073 } } peers { permanent_uuid: "a78f88301ed54f48bf9e17f58fd62d5e" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 41103 } } peers { permanent_uuid: "9a848619566249798e7b9f2d4118c664" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 41397 } }
I20260812 02:39:45.465132 25759 ts_tablet_manager.cc:1428] T 5963125c6453469db15e9c9b183b1856 P a78f88301ed54f48bf9e17f58fd62d5e: Time spent starting tablet: real 0.015s user 0.010s sys 0.003s
I20260812 02:39:45.537870 25762 raft_consensus.cc:482] T 5963125c6453469db15e9c9b183b1856 P 7c0a7ebfc3e74adeb520daaaf7609f27 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 02:39:45.538287 25762 raft_consensus.cc:504] T 5963125c6453469db15e9c9b183b1856 P 7c0a7ebfc3e74adeb520daaaf7609f27 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0a7ebfc3e74adeb520daaaf7609f27" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 46073 } } peers { permanent_uuid: "a78f88301ed54f48bf9e17f58fd62d5e" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 41103 } } peers { permanent_uuid: "9a848619566249798e7b9f2d4118c664" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 41397 } }
I20260812 02:39:45.540153 25762 leader_election.cc:288] T 5963125c6453469db15e9c9b183b1856 P 7c0a7ebfc3e74adeb520daaaf7609f27 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a78f88301ed54f48bf9e17f58fd62d5e (127.7.132.129:41103), 9a848619566249798e7b9f2d4118c664 (127.7.132.130:41397)
I20260812 02:39:45.545966 25753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:45.540827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:49603 (local address 127.7.132.130:41397)
0812 02:39:45.541030 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0812 02:39:45.541037 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:39:45.541070 (+ 33us) server_negotiation.cc:407] Connection header received
0812 02:39:45.541149 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:39:45.541156 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:39:45.541238 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:39:45.541363 (+ 125us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:39:45.541751 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:45.542690 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:45.544306 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:45.545086 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:45.545245 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:39:45.545320 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 02:39:45.545589 (+ 269us) server_negotiation.cc:299] Negotiation successful
0812 02:39:45.545712 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 02:39:45.545998 25739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:45.540417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.130:41397 (local address 127.7.132.131:49603)
0812 02:39:45.540643 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0812 02:39:45.540665 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 02:39:45.540814 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:39:45.541388 (+ 574us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:39:45.541392 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:39:45.541421 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:39:45.541604 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:39:45.541613 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:39:45.542817 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:39:45.542824 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:39:45.544152 (+ 1328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:39:45.544164 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:39:45.545246 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:39:45.545252 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:39:45.545427 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:39:45.545451 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 02:39:45.545560 (+ 109us) client_negotiation.cc:240] Negotiation successful
0812 02:39:45.545696 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60,"mutex_wait_us":1}
I20260812 02:39:45.546672 25749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:45.541001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:54643 (local address 127.7.132.129:41103)
0812 02:39:45.541199 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0812 02:39:45.541206 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:39:45.541495 (+ 289us) server_negotiation.cc:407] Connection header received
0812 02:39:45.541718 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:39:45.541721 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:39:45.541780 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:39:45.541879 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:39:45.542460 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:45.543158 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:45.544879 (+ 1721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:45.545659 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:45.545845 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:39:45.545914 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 02:39:45.546289 (+ 375us) server_negotiation.cc:299] Negotiation successful
0812 02:39:45.546418 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260812 02:39:45.547106 25526 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5963125c6453469db15e9c9b183b1856" candidate_uuid: "7c0a7ebfc3e74adeb520daaaf7609f27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a848619566249798e7b9f2d4118c664" is_pre_election: true
I20260812 02:39:45.547447 25765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:45.540605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:41103 (local address 127.7.132.131:54643)
0812 02:39:45.541383 (+ 778us) negotiation.cc:107] Waiting for socket to connect
0812 02:39:45.541397 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 02:39:45.541578 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:39:45.541901 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:39:45.541906 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:39:45.541929 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:39:45.542311 (+ 382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:39:45.542321 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:39:45.543310 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:39:45.543315 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:39:45.544719 (+ 1404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:39:45.544732 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:39:45.545861 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:39:45.545871 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:39:45.546061 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:39:45.546133 (+ 72us) client_negotiation.cc:769] Sending connection context
0812 02:39:45.546868 (+ 735us) client_negotiation.cc:240] Negotiation successful
0812 02:39:45.547048 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":654,"mutex_wait_us":23,"thread_start_us":490,"threads_started":1}
I20260812 02:39:45.547709 25526 raft_consensus.cc:2454] T 5963125c6453469db15e9c9b183b1856 P 9a848619566249798e7b9f2d4118c664 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c0a7ebfc3e74adeb520daaaf7609f27 in term 0.
I20260812 02:39:45.548364 25600 leader_election.cc:302] T 5963125c6453469db15e9c9b183b1856 P 7c0a7ebfc3e74adeb520daaaf7609f27 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c0a7ebfc3e74adeb520daaaf7609f27, 9a848619566249798e7b9f2d4118c664; no voters:
I20260812 02:39:45.548767 25762 raft_consensus.cc:2781] T 5963125c6453469db15e9c9b183b1856 P 7c0a7ebfc3e74adeb520daaaf7609f27 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 02:39:45.548566 25378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5963125c6453469db15e9c9b183b1856" candidate_uuid: "7c0a7ebfc3e74adeb520daaaf7609f27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a78f88301ed54f48bf9e17f58fd62d5e" is_pre_election: true
I20260812 02:39:45.548926 25762 raft_consensus.cc:482] T 5963125c6453469db15e9c9b183b1856 P 7c0a7ebfc3e74adeb520daaaf7609f27 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 02:39:45.549054 25762 raft_consensus.cc:3037] T 5963125c6453469db15e9c9b183b1856 P 7c0a7ebfc3e74adeb520daaaf7609f27 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:39:45.549149 25378 raft_consensus.cc:2454] T 5963125c6453469db15e9c9b183b1856 P a78f88301ed54f48bf9e17f58fd62d5e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c0a7ebfc3e74adeb520daaaf7609f27 in term 0.
I20260812 02:39:45.551633 25762 raft_consensus.cc:504] T 5963125c6453469db15e9c9b183b1856 P 7c0a7ebfc3e74adeb520daaaf7609f27 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0a7ebfc3e74adeb520daaaf7609f27" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 46073 } } peers { permanent_uuid: "a78f88301ed54f48bf9e17f58fd62d5e" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 41103 } } peers { permanent_uuid: "9a848619566249798e7b9f2d4118c664" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 41397 } }
I20260812 02:39:45.552258 25762 leader_election.cc:288] T 5963125c6453469db15e9c9b183b1856 P 7c0a7ebfc3e74adeb520daaaf7609f27 [CANDIDATE]: Term 1 election: Requested vote from peers a78f88301ed54f48bf9e17f58fd62d5e (127.7.132.129:41103), 9a848619566249798e7b9f2d4118c664 (127.7.132.130:41397)
I20260812 02:39:45.552664 25378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5963125c6453469db15e9c9b183b1856" candidate_uuid: "7c0a7ebfc3e74adeb520daaaf7609f27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a78f88301ed54f48bf9e17f58fd62d5e"
I20260812 02:39:45.552798 25526 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5963125c6453469db15e9c9b183b1856" candidate_uuid: "7c0a7ebfc3e74adeb520daaaf7609f27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a848619566249798e7b9f2d4118c664"
I20260812 02:39:45.552883 25378 raft_consensus.cc:3037] T 5963125c6453469db15e9c9b183b1856 P a78f88301ed54f48bf9e17f58fd62d5e [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:39:45.552994 25526 raft_consensus.cc:3037] T 5963125c6453469db15e9c9b183b1856 P 9a848619566249798e7b9f2d4118c664 [term 0 FOLLOWER]: Advancing to term 1
W20260812 02:39:45.556186 25736 tablet.cc:2369] T 5963125c6453469db15e9c9b183b1856 P 7c0a7ebfc3e74adeb520daaaf7609f27: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 02:39:45.556196 25378 raft_consensus.cc:2454] T 5963125c6453469db15e9c9b183b1856 P a78f88301ed54f48bf9e17f58fd62d5e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c0a7ebfc3e74adeb520daaaf7609f27 in term 1.
I20260812 02:39:45.556196 25526 raft_consensus.cc:2454] T 5963125c6453469db15e9c9b183b1856 P 9a848619566249798e7b9f2d4118c664 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c0a7ebfc3e74adeb520daaaf7609f27 in term 1.
I20260812 02:39:45.556599 25604 leader_election.cc:302] T 5963125c6453469db15e9c9b183b1856 P 7c0a7ebfc3e74adeb520daaaf7609f27 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c0a7ebfc3e74adeb520daaaf7609f27, a78f88301ed54f48bf9e17f58fd62d5e; no voters:
I20260812 02:39:45.556941 25762 raft_consensus.cc:2781] T 5963125c6453469db15e9c9b183b1856 P 7c0a7ebfc3e74adeb520daaaf7609f27 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:39:45.557420 25762 raft_consensus.cc:686] T 5963125c6453469db15e9c9b183b1856 P 7c0a7ebfc3e74adeb520daaaf7609f27 [term 1 LEADER]: Becoming Leader. State: Replica: 7c0a7ebfc3e74adeb520daaaf7609f27, State: Running, Role: LEADER
I20260812 02:39:45.557883 25762 consensus_queue.cc:237] T 5963125c6453469db15e9c9b183b1856 P 7c0a7ebfc3e74adeb520daaaf7609f27 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0a7ebfc3e74adeb520daaaf7609f27" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 46073 } } peers { permanent_uuid: "a78f88301ed54f48bf9e17f58fd62d5e" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 41103 } } peers { permanent_uuid: "9a848619566249798e7b9f2d4118c664" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 41397 } }
I20260812 02:39:45.562798 25215 catalog_manager.cc:5495] T 5963125c6453469db15e9c9b183b1856 P 7c0a7ebfc3e74adeb520daaaf7609f27 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c0a7ebfc3e74adeb520daaaf7609f27 (127.7.132.131). New cstate: current_term: 1 leader_uuid: "7c0a7ebfc3e74adeb520daaaf7609f27" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0a7ebfc3e74adeb520daaaf7609f27" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 46073 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a78f88301ed54f48bf9e17f58fd62d5e" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 41103 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a848619566249798e7b9f2d4118c664" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 41397 } health_report { overall_health: UNKNOWN } } }
W20260812 02:39:45.585467 25579 tablet.cc:2369] T 5963125c6453469db15e9c9b183b1856 P 9a848619566249798e7b9f2d4118c664: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 02:39:45.609954 25750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:45.604080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56794 (local address 127.7.132.131:46073)
0812 02:39:45.604299 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0812 02:39:45.604306 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:39:45.604413 (+ 107us) server_negotiation.cc:407] Connection header received
0812 02:39:45.604677 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:39:45.604682 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:39:45.604773 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:39:45.604932 (+ 159us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 02:39:45.605347 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:45.606171 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:45.607700 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:45.608106 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:45.608246 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:39:45.608768 (+ 522us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 02:39:45.609302 (+ 534us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 02:39:45.609425 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0812 02:39:45.609570 (+ 145us) server_negotiation.cc:299] Negotiation successful
0812 02:39:45.609695 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
W20260812 02:39:45.610761 25429 tablet.cc:2369] T 5963125c6453469db15e9c9b183b1856 P a78f88301ed54f48bf9e17f58fd62d5e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 02:39:45.618535 25378 raft_consensus.cc:1261] T 5963125c6453469db15e9c9b183b1856 P a78f88301ed54f48bf9e17f58fd62d5e [term 1 FOLLOWER]: Refusing update from remote peer 7c0a7ebfc3e74adeb520daaaf7609f27: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:39:45.618533 25526 raft_consensus.cc:1261] T 5963125c6453469db15e9c9b183b1856 P 9a848619566249798e7b9f2d4118c664 [term 1 FOLLOWER]: Refusing update from remote peer 7c0a7ebfc3e74adeb520daaaf7609f27: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:39:45.619591 25762 consensus_queue.cc:1035] T 5963125c6453469db15e9c9b183b1856 P 7c0a7ebfc3e74adeb520daaaf7609f27 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a848619566249798e7b9f2d4118c664" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 41397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:39:45.619894 25766 consensus_queue.cc:1035] T 5963125c6453469db15e9c9b183b1856 P 7c0a7ebfc3e74adeb520daaaf7609f27 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a78f88301ed54f48bf9e17f58fd62d5e" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 41103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:39:45.634222 25771 mvcc.cc:205] Tried to move back new op lower bound from 7317513771482820608 to 7317513771248848896. Current Snapshot: MvccSnapshot[applied={T|T < 7317513771482820608}]
I20260812 02:39:45.648823 25215 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:42736:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 02:39:45.649184 25215 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:42736:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 02:39:45.653288 25215 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ce6e17ddb54e4dc39dc6f8ef9c7f2d94: Sending DeleteTablet for 3 replicas of tablet 5963125c6453469db15e9c9b183b1856
I20260812 02:39:45.654498 25656 tablet_service.cc:1514] Processing DeleteTablet for tablet 5963125c6453469db15e9c9b183b1856 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:39:45 UTC) from {username='kudu', principal='kudu/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:56790
I20260812 02:39:45.654667 25506 tablet_service.cc:1514] Processing DeleteTablet for tablet 5963125c6453469db15e9c9b183b1856 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:39:45 UTC) from {username='kudu', principal='kudu/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:55650
I20260812 02:39:45.654696 25354 tablet_service.cc:1514] Processing DeleteTablet for tablet 5963125c6453469db15e9c9b183b1856 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:39:45 UTC) from {username='kudu', principal='kudu/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:40142
I20260812 02:39:45.656605 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 25259
I20260812 02:39:45.665820 25782 tablet_replica.cc:330] T 5963125c6453469db15e9c9b183b1856 P a78f88301ed54f48bf9e17f58fd62d5e: stopping tablet replica
I20260812 02:39:45.666621 25782 raft_consensus.cc:2229] T 5963125c6453469db15e9c9b183b1856 P a78f88301ed54f48bf9e17f58fd62d5e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 02:39:45.667362 25782 raft_consensus.cc:2258] T 5963125c6453469db15e9c9b183b1856 P a78f88301ed54f48bf9e17f58fd62d5e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:39:45.668064 25781 tablet_replica.cc:330] T 5963125c6453469db15e9c9b183b1856 P 9a848619566249798e7b9f2d4118c664: stopping tablet replica
I20260812 02:39:45.668892 25779 tablet_replica.cc:330] T 5963125c6453469db15e9c9b183b1856 P 7c0a7ebfc3e74adeb520daaaf7609f27: stopping tablet replica
I20260812 02:39:45.669042 25781 raft_consensus.cc:2229] T 5963125c6453469db15e9c9b183b1856 P 9a848619566249798e7b9f2d4118c664 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 02:39:45.669365 25779 raft_consensus.cc:2229] T 5963125c6453469db15e9c9b183b1856 P 7c0a7ebfc3e74adeb520daaaf7609f27 [term 1 LEADER]: Raft consensus shutting down.
I20260812 02:39:45.669701 25781 raft_consensus.cc:2258] T 5963125c6453469db15e9c9b183b1856 P 9a848619566249798e7b9f2d4118c664 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:39:45.670120 25779 raft_consensus.cc:2258] T 5963125c6453469db15e9c9b183b1856 P 7c0a7ebfc3e74adeb520daaaf7609f27 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:39:45.670742 25782 ts_tablet_manager.cc:1905] T 5963125c6453469db15e9c9b183b1856 P a78f88301ed54f48bf9e17f58fd62d5e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:39:45.672240 25781 ts_tablet_manager.cc:1905] T 5963125c6453469db15e9c9b183b1856 P 9a848619566249798e7b9f2d4118c664: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:39:45.679258 25782 ts_tablet_manager.cc:1918] T 5963125c6453469db15e9c9b183b1856 P a78f88301ed54f48bf9e17f58fd62d5e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 02:39:45.679581 25782 log.cc:1196] T 5963125c6453469db15e9c9b183b1856 P a78f88301ed54f48bf9e17f58fd62d5e: Deleting WAL directory at /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-0/wal/wals/5963125c6453469db15e9c9b183b1856
I20260812 02:39:45.680265 25782 ts_tablet_manager.cc:1939] T 5963125c6453469db15e9c9b183b1856 P a78f88301ed54f48bf9e17f58fd62d5e: Deleting consensus metadata
I20260812 02:39:45.680387 25781 ts_tablet_manager.cc:1918] T 5963125c6453469db15e9c9b183b1856 P 9a848619566249798e7b9f2d4118c664: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 02:39:45.680661 25781 log.cc:1196] T 5963125c6453469db15e9c9b183b1856 P 9a848619566249798e7b9f2d4118c664: Deleting WAL directory at /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-1/wal/wals/5963125c6453469db15e9c9b183b1856
I20260812 02:39:45.681363 25781 ts_tablet_manager.cc:1939] T 5963125c6453469db15e9c9b183b1856 P 9a848619566249798e7b9f2d4118c664: Deleting consensus metadata
I20260812 02:39:45.682502 25200 catalog_manager.cc:4841] TS a78f88301ed54f48bf9e17f58fd62d5e (127.7.132.129:41103): tablet 5963125c6453469db15e9c9b183b1856 (table test-table [id=13107de8e5a048c08606d96655df38ca]) successfully deleted
I20260812 02:39:45.682570 25779 ts_tablet_manager.cc:1905] T 5963125c6453469db15e9c9b183b1856 P 7c0a7ebfc3e74adeb520daaaf7609f27: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:39:45.683964 25197 catalog_manager.cc:4841] TS 9a848619566249798e7b9f2d4118c664 (127.7.132.130:41397): tablet 5963125c6453469db15e9c9b183b1856 (table test-table [id=13107de8e5a048c08606d96655df38ca]) successfully deleted
I20260812 02:39:45.684343 25749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:45.657536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40144 (local address 127.7.132.129:41103)
0812 02:39:45.657722 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0812 02:39:45.657729 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:39:45.658128 (+ 399us) server_negotiation.cc:407] Connection header received
0812 02:39:45.658213 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:39:45.658221 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:39:45.658335 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:39:45.658444 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:39:45.661168 (+ 2724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:45.662810 (+ 1642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:45.665485 (+ 2675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:45.666532 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:45.666703 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:39:45.675358 (+ 8655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:39:45.675375 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:39:45.675391 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:39:45.675446 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:39:45.679678 (+ 4232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:39:45.680783 (+ 1105us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:39:45.680793 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:39:45.680796 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:39:45.680978 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:39:45.681457 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:39:45.681462 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:39:45.681465 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:39:45.681971 (+ 506us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:39:45.682325 (+ 354us) server_negotiation.cc:1035] Waiting for connection context
0812 02:39:45.683696 (+ 1371us) server_negotiation.cc:299] Negotiation successful
0812 02:39:45.683932 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 02:39:45.685037 25423 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:39:45.691360 25779 ts_tablet_manager.cc:1918] T 5963125c6453469db15e9c9b183b1856 P 7c0a7ebfc3e74adeb520daaaf7609f27: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 02:39:45.691565 25779 log.cc:1196] T 5963125c6453469db15e9c9b183b1856 P 7c0a7ebfc3e74adeb520daaaf7609f27: Deleting WAL directory at /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-2/wal/wals/5963125c6453469db15e9c9b183b1856
I20260812 02:39:45.692170 25779 ts_tablet_manager.cc:1939] T 5963125c6453469db15e9c9b183b1856 P 7c0a7ebfc3e74adeb520daaaf7609f27: Deleting consensus metadata
I20260812 02:39:45.694200 25199 catalog_manager.cc:4841] TS 7c0a7ebfc3e74adeb520daaaf7609f27 (127.7.132.131:46073): tablet 5963125c6453469db15e9c9b183b1856 (table test-table [id=13107de8e5a048c08606d96655df38ca]) successfully deleted
I20260812 02:39:45.777446 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 25259
I20260812 02:39:45.802206 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 25432
I20260812 02:39:45.817389 25753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:45.803044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55658 (local address 127.7.132.130:41397)
0812 02:39:45.803244 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0812 02:39:45.803252 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:39:45.803350 (+ 98us) server_negotiation.cc:407] Connection header received
0812 02:39:45.803585 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:39:45.803590 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:39:45.803660 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:39:45.803769 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:39:45.805226 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:45.805975 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:45.807499 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:45.808094 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:45.808225 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:39:45.811383 (+ 3158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:39:45.811400 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:39:45.811416 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:39:45.811449 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:39:45.814441 (+ 2992us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:39:45.815179 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:39:45.815195 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:39:45.815197 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:39:45.815376 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:39:45.815773 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:39:45.815778 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:39:45.815782 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:39:45.816160 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:39:45.816477 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0812 02:39:45.816804 (+ 327us) server_negotiation.cc:299] Negotiation successful
0812 02:39:45.817029 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260812 02:39:45.818768 25572 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:39:45.916657 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 25432
I20260812 02:39:45.937825 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 25581
I20260812 02:39:45.954772 25750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:45.938882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56796 (local address 127.7.132.131:46073)
0812 02:39:45.939102 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0812 02:39:45.939108 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:39:45.939132 (+ 24us) server_negotiation.cc:407] Connection header received
0812 02:39:45.939275 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:39:45.939283 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:39:45.939378 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:39:45.939501 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:39:45.941036 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:45.942481 (+ 1445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:45.943843 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:39:45.944573 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:39:45.944723 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:39:45.947466 (+ 2743us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:39:45.947481 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:39:45.947501 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:39:45.947550 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:39:45.951659 (+ 4109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:39:45.952390 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:39:45.952400 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:39:45.952404 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:39:45.952609 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:39:45.952987 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:39:45.952993 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:39:45.952996 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:39:45.953488 (+ 492us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:39:45.953806 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0812 02:39:45.954070 (+ 264us) server_negotiation.cc:299] Negotiation successful
0812 02:39:45.954341 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 02:39:45.955595 25729 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:39:46.079870 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 25581
I20260812 02:39:46.104727 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 25178
I20260812 02:39:46.105854 25252 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:39:46.222259 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 25178
I20260812 02:39:46.250937 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:44921
--webserver_interface=127.7.132.190
--webserver_port=45015
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:42213
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.132.190:44921
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:39:46.610374 25788 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:39:46.610661 25788 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:39:46.610836 25788 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:39:46.618273 25788 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:39:46.618448 25788 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:39:46.618543 25788 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:39:46.618616 25788 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:39:46.618661 25788 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:39:46.628556 25788 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:42213
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.132.190:44921
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:44921
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.132.190
--webserver_port=45015
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:39:46.629873 25788 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:39:46.631210 25788 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:39:46.643198 25793 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:39:46.643242 25794 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:39:46.643154 25796 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:39:46.644335 25788 server_base.cc:1034] running on GCE node
I20260812 02:39:46.645066 25788 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:39:46.646502 25788 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:39:46.647742 25788 hybrid_clock.cc:648] HybridClock initialized: now 1786502386647688 us; error 61 us; skew 500 ppm
I20260812 02:39:46.648111 25788 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:39:46 dist-test-slave-mn4r krb5kdc[24530](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502386, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:39:46.651250 25788 init.cc:377] Logged in from keytab as kudu/127.7.132.190@KRBTEST.COM (short username kudu)
I20260812 02:39:46.652819 25788 webserver.cc:460] Webserver started at http://127.7.132.190:45015/ using document root <none> and password file <none>
I20260812 02:39:46.653534 25788 fs_manager.cc:359] Metadata directory not provided
I20260812 02:39:46.653685 25788 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:39:46.659235 25788 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 02:39:46.661985 25807 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:39:46.663381 25788 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 02:39:46.663573 25788 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/master-0/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/master-0/wal
uuid: "ce6e17ddb54e4dc39dc6f8ef9c7f2d94"
format_stamp: "Formatted at 2026-08-12 02:39:40 on dist-test-slave-mn4r"
server_key: "9ba3905bf917b4c9aa9d00b47d3523df"
server_key_iv: "0338562476a92610dbf99c24a56e6b70"
server_key_version: "encryptionkey@0"
I20260812 02:39:46.664584 25788 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:39:46.681569 25788 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:39:46.685734 25788 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:39:46.686034 25788 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:39:46.705463 25788 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.190:44921
I20260812 02:39:46.705471 25865 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.190:44921 every 8 connection(s)
I20260812 02:39:46.707084 25788 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
I20260812 02:39:46.709908 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 25788
I20260812 02:39:46.711018 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.129:41103
--local_ip_for_outbound_sockets=127.7.132.129
--tserver_master_addrs=127.7.132.190:44921
--webserver_port=34017
--webserver_interface=127.7.132.129
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.129.keytab
--principal=kudu/127.7.132.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:42213
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 02:39:46.716609 25867 sys_catalog.cc:263] Verifying existing consensus state
I20260812 02:39:46.721993 25867 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce6e17ddb54e4dc39dc6f8ef9c7f2d94: Bootstrap starting.
I20260812 02:39:46.743225 25867 log.cc:824] T 00000000000000000000000000000000 P ce6e17ddb54e4dc39dc6f8ef9c7f2d94: Log is configured to *not* fsync() on all Append() calls
I20260812 02:39:46.766806 25867 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce6e17ddb54e4dc39dc6f8ef9c7f2d94: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 02:39:46.767411 25867 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce6e17ddb54e4dc39dc6f8ef9c7f2d94: Bootstrap complete.
I20260812 02:39:46.777177 25867 raft_consensus.cc:348] T 00000000000000000000000000000000 P ce6e17ddb54e4dc39dc6f8ef9c7f2d94 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce6e17ddb54e4dc39dc6f8ef9c7f2d94" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 44921 } }
I20260812 02:39:46.778148 25867 raft_consensus.cc:729] T 00000000000000000000000000000000 P ce6e17ddb54e4dc39dc6f8ef9c7f2d94 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce6e17ddb54e4dc39dc6f8ef9c7f2d94, State: Initialized, Role: FOLLOWER
I20260812 02:39:46.778837 25867 consensus_queue.cc:260] T 00000000000000000000000000000000 P ce6e17ddb54e4dc39dc6f8ef9c7f2d94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce6e17ddb54e4dc39dc6f8ef9c7f2d94" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 44921 } }
I20260812 02:39:46.779085 25867 raft_consensus.cc:388] T 00000000000000000000000000000000 P ce6e17ddb54e4dc39dc6f8ef9c7f2d94 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 02:39:46.779207 25867 raft_consensus.cc:482] T 00000000000000000000000000000000 P ce6e17ddb54e4dc39dc6f8ef9c7f2d94 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 02:39:46.779412 25867 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ce6e17ddb54e4dc39dc6f8ef9c7f2d94 [term 2 FOLLOWER]: Advancing to term 3
I20260812 02:39:46.783324 25867 raft_consensus.cc:504] T 00000000000000000000000000000000 P ce6e17ddb54e4dc39dc6f8ef9c7f2d94 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce6e17ddb54e4dc39dc6f8ef9c7f2d94" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 44921 } }
I20260812 02:39:46.783855 25867 leader_election.cc:302] T 00000000000000000000000000000000 P ce6e17ddb54e4dc39dc6f8ef9c7f2d94 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ce6e17ddb54e4dc39dc6f8ef9c7f2d94; no voters:
I20260812 02:39:46.784551 25867 leader_election.cc:288] T 00000000000000000000000000000000 P ce6e17ddb54e4dc39dc6f8ef9c7f2d94 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260812 02:39:46.784711 25871 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ce6e17ddb54e4dc39dc6f8ef9c7f2d94 [term 3 FOLLOWER]: Leader election won for term 3
I20260812 02:39:46.785868 25871 raft_consensus.cc:686] T 00000000000000000000000000000000 P ce6e17ddb54e4dc39dc6f8ef9c7f2d94 [term 3 LEADER]: Becoming Leader. State: Replica: ce6e17ddb54e4dc39dc6f8ef9c7f2d94, State: Running, Role: LEADER
I20260812 02:39:46.786590 25871 consensus_queue.cc:237] T 00000000000000000000000000000000 P ce6e17ddb54e4dc39dc6f8ef9c7f2d94 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce6e17ddb54e4dc39dc6f8ef9c7f2d94" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 44921 } }
I20260812 02:39:46.787000 25867 sys_catalog.cc:564] T 00000000000000000000000000000000 P ce6e17ddb54e4dc39dc6f8ef9c7f2d94 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:39:46.789572 25873 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce6e17ddb54e4dc39dc6f8ef9c7f2d94 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ce6e17ddb54e4dc39dc6f8ef9c7f2d94. Latest consensus state: current_term: 3 leader_uuid: "ce6e17ddb54e4dc39dc6f8ef9c7f2d94" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce6e17ddb54e4dc39dc6f8ef9c7f2d94" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 44921 } } }
I20260812 02:39:46.789563 25872 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce6e17ddb54e4dc39dc6f8ef9c7f2d94 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "ce6e17ddb54e4dc39dc6f8ef9c7f2d94" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce6e17ddb54e4dc39dc6f8ef9c7f2d94" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 44921 } } }
I20260812 02:39:46.790170 25872 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce6e17ddb54e4dc39dc6f8ef9c7f2d94 [sys.catalog]: This master's current role is: LEADER
I20260812 02:39:46.790256 25873 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce6e17ddb54e4dc39dc6f8ef9c7f2d94 [sys.catalog]: This master's current role is: LEADER
I20260812 02:39:46.801204 25878 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:39:46.810837 25878 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 02:39:46.812588 25878 catalog_manager.cc:1259] Loaded cluster ID: a19eed9dfb2546c89c27bf5be3380eb7
I20260812 02:39:46.812784 25878 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260812 02:39:46.815459 25878 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f23c1f4e9ad google::LogMessage::Fail() at ??:0
@ 0x7f23c1f5188a google::LogMessage::SendToLog() at ??:0
@ 0x7f23c1f4e4de google::LogMessage::Flush() at ??:0
@ 0x7f23c1f4fa19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f23d7981512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f23d797e976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f23d7a06610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f23d7a06439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f23c4b9a345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f23c4b9d91c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f23c4b9d799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f23c4b78eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f23c7e526db start_thread at ??:0
@ 0x7f23c0de861f clone at ??:0
W20260812 02:39:47.144797 25869 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:39:47.145097 25869 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:39:47.145265 25869 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:39:47.152804 25869 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:39:47.152920 25869 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:39:47.153086 25869 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.129
I20260812 02:39:47.164022 25869 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:42213
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.129.keytab
--principal=kudu/127.7.132.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.129:41103
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.132.129
--webserver_port=34017
--tserver_master_addrs=127.7.132.190:44921
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.129
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:39:47.165365 25869 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:39:47.166865 25869 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:39:47.180356 25912 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:39:47.180387 25915 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:39:47.180322 25913 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:39:47.181567 25869 server_base.cc:1034] running on GCE node
I20260812 02:39:47.182331 25869 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:39:47.183331 25869 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:39:47.184599 25869 hybrid_clock.cc:648] HybridClock initialized: now 1786502387184539 us; error 47 us; skew 500 ppm
I20260812 02:39:47.184973 25869 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:39:47 dist-test-slave-mn4r krb5kdc[24530](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502387, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:39:47.188251 25869 init.cc:377] Logged in from keytab as kudu/127.7.132.129@KRBTEST.COM (short username kudu)
I20260812 02:39:47.189847 25869 webserver.cc:460] Webserver started at http://127.7.132.129:34017/ using document root <none> and password file <none>
I20260812 02:39:47.190675 25869 fs_manager.cc:359] Metadata directory not provided
I20260812 02:39:47.190826 25869 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:39:47.196383 25869 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260812 02:39:47.199333 25923 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:39:47.201390 25869 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260812 02:39:47.201601 25869 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-0/wal
uuid: "a78f88301ed54f48bf9e17f58fd62d5e"
format_stamp: "Formatted at 2026-08-12 02:39:41 on dist-test-slave-mn4r"
server_key: "7b5f87b6450b31d2c4b0e385768e0548"
server_key_iv: "90cbe381b2dc9e8fe4cac009c9028cfa"
server_key_version: "encryptionkey@0"
I20260812 02:39:47.203016 25869 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:39:47.218389 25869 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:39:47.222412 25869 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:39:47.222752 25869 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:39:47.223778 25869 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:39:47.225131 25869 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:39:47.225219 25869 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:39:47.225364 25869 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:39:47.225466 25869 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:39:47.260411 25869 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.129:41103
I20260812 02:39:47.260493 26052 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.129:41103 every 8 connection(s)
I20260812 02:39:47.262013 25869 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-0/data/info.pb
I20260812 02:39:47.263917 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 25869
I20260812 02:39:47.264622 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.130:41397
--local_ip_for_outbound_sockets=127.7.132.130
--tserver_master_addrs=127.7.132.190:44921
--webserver_port=36867
--webserver_interface=127.7.132.130
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.130.keytab
--principal=kudu/127.7.132.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:42213
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 02:39:47.269518 26055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:47.267025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:44921 (local address 127.7.132.129:53011)
0812 02:39:47.267938 (+ 913us) negotiation.cc:107] Waiting for socket to connect
0812 02:39:47.268223 (+ 285us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:44921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":563,"thread_start_us":325,"threads_started":1}
W20260812 02:39:47.270354 26053 heartbeater.cc:644] Failed to heartbeat to 127.7.132.190:44921 (0 consecutive failures): Network error: Failed to ping master at 127.7.132.190:44921: Client connection negotiation failed: client connection to 127.7.132.190:44921: connect: Connection refused (error 111)
I20260812 02:39:47.271374 26055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:47.270916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:44921 (local address 127.7.132.129:51395)
0812 02:39:47.271112 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0812 02:39:47.271211 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:44921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260812 02:39:47.272365 26055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:47.271944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:44921 (local address 127.7.132.129:50515)
0812 02:39:47.272112 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0812 02:39:47.272213 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:44921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260812 02:39:47.272642 26053 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 02:39:47.620659 26056 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:39:47.621061 26056 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:39:47.621217 26056 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:39:47.628662 26056 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:39:47.628764 26056 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:39:47.628965 26056 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.130
I20260812 02:39:47.638800 26056 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:42213
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.130.keytab
--principal=kudu/127.7.132.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.130:41397
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.132.130
--webserver_port=36867
--tserver_master_addrs=127.7.132.190:44921
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.130
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:39:47.640179 26056 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:39:47.641582 26056 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:39:47.655447 26062 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:39:47.655529 26064 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:39:47.655535 26061 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:39:47.656729 26056 server_base.cc:1034] running on GCE node
I20260812 02:39:47.657410 26056 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:39:47.658488 26056 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:39:47.659719 26056 hybrid_clock.cc:648] HybridClock initialized: now 1786502387659669 us; error 46 us; skew 500 ppm
I20260812 02:39:47.660048 26056 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:39:47 dist-test-slave-mn4r krb5kdc[24530](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502387, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:39:47.663375 26056 init.cc:377] Logged in from keytab as kudu/127.7.132.130@KRBTEST.COM (short username kudu)
I20260812 02:39:47.664801 26056 webserver.cc:460] Webserver started at http://127.7.132.130:36867/ using document root <none> and password file <none>
I20260812 02:39:47.665518 26056 fs_manager.cc:359] Metadata directory not provided
I20260812 02:39:47.665668 26056 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:39:47.671139 26056 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 02:39:47.673954 26071 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:39:47.675390 26056 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 02:39:47.675572 26056 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-1/wal
uuid: "9a848619566249798e7b9f2d4118c664"
format_stamp: "Formatted at 2026-08-12 02:39:41 on dist-test-slave-mn4r"
server_key: "8d3147d5c2e5775b5e5295a7d3f1933e"
server_key_iv: "caf67665118634047b0df87a316228e8"
server_key_version: "encryptionkey@0"
I20260812 02:39:47.676568 26056 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:39:47.696791 26056 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:39:47.700997 26056 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:39:47.701290 26056 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:39:47.702410 26056 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:39:47.703816 26056 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:39:47.703943 26056 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:39:47.704089 26056 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:39:47.704214 26056 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:39:47.738601 26056 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.130:41397
I20260812 02:39:47.738690 26202 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.130:41397 every 8 connection(s)
I20260812 02:39:47.740139 26056 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-1/data/info.pb
I20260812 02:39:47.744894 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 26056
I20260812 02:39:47.744961 26205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:47.743157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:44921 (local address 127.7.132.130:48135)
0812 02:39:47.743875 (+ 718us) negotiation.cc:107] Waiting for socket to connect
0812 02:39:47.744092 (+ 217us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:44921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":270,"threads_started":1}
W20260812 02:39:47.745596 26203 heartbeater.cc:644] Failed to heartbeat to 127.7.132.190:44921 (0 consecutive failures): Network error: Failed to ping master at 127.7.132.190:44921: Client connection negotiation failed: client connection to 127.7.132.190:44921: connect: Connection refused (error 111)
I20260812 02:39:47.745750 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.131:46073
--local_ip_for_outbound_sockets=127.7.132.131
--tserver_master_addrs=127.7.132.190:44921
--webserver_port=41163
--webserver_interface=127.7.132.131
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.131.keytab
--principal=kudu/127.7.132.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:42213
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 02:39:47.746490 26205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:47.746176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:44921 (local address 127.7.132.130:37067)
0812 02:39:47.746318 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0812 02:39:47.746391 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:44921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260812 02:39:47.747326 26205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:47.746995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:44921 (local address 127.7.132.130:60177)
0812 02:39:47.747148 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0812 02:39:47.747225 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:44921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260812 02:39:47.747550 26203 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 02:39:48.090487 26206 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:39:48.090795 26206 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:39:48.090891 26206 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:39:48.098466 26206 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:39:48.098599 26206 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:39:48.098713 26206 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.131
I20260812 02:39:48.109099 26206 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:42213
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.131.keytab
--principal=kudu/127.7.132.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.131:46073
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.132.131
--webserver_port=41163
--tserver_master_addrs=127.7.132.190:44921
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.131
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:39:48.110498 26206 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:39:48.111832 26206 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:39:48.125514 26215 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:39:48.125574 26213 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:39:48.125571 26212 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:39:48.126468 26206 server_base.cc:1034] running on GCE node
I20260812 02:39:48.127090 26206 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:39:48.128049 26206 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:39:48.129280 26206 hybrid_clock.cc:648] HybridClock initialized: now 1786502388129233 us; error 57 us; skew 500 ppm
I20260812 02:39:48.129603 26206 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:39:48 dist-test-slave-mn4r krb5kdc[24530](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502388, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:39:48.132825 26206 init.cc:377] Logged in from keytab as kudu/127.7.132.131@KRBTEST.COM (short username kudu)
I20260812 02:39:48.134382 26206 webserver.cc:460] Webserver started at http://127.7.132.131:41163/ using document root <none> and password file <none>
I20260812 02:39:48.135056 26206 fs_manager.cc:359] Metadata directory not provided
I20260812 02:39:48.135155 26206 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:39:48.140583 26206 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.000s sys 0.005s
I20260812 02:39:48.143431 26225 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:39:48.144869 26206 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 02:39:48.145013 26206 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-2/wal
uuid: "7c0a7ebfc3e74adeb520daaaf7609f27"
format_stamp: "Formatted at 2026-08-12 02:39:42 on dist-test-slave-mn4r"
server_key: "82d319a99df0be3d1797d77d9546b6aa"
server_key_iv: "763557f03a3893646fb09d441e17cf97"
server_key_version: "encryptionkey@0"
I20260812 02:39:48.145992 26206 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:39:48.161988 26206 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:39:48.165853 26206 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:39:48.166121 26206 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:39:48.166992 26206 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:39:48.168246 26206 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:39:48.168330 26206 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:39:48.168406 26206 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:39:48.168465 26206 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:39:48.202625 26206 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.131:46073
I20260812 02:39:48.202566 26359 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.131:46073 every 8 connection(s)
I20260812 02:39:48.204663 26206 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786502156971023-7698-0/minicluster-data/ts-2/data/info.pb
I20260812 02:39:48.205403 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 26206
I20260812 02:39:48.210397 26364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:48.208409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:44921 (local address 127.7.132.131:59457)
0812 02:39:48.209126 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0812 02:39:48.209342 (+ 216us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:44921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":273,"threads_started":1}
W20260812 02:39:48.211030 26361 heartbeater.cc:644] Failed to heartbeat to 127.7.132.190:44921 (0 consecutive failures): Network error: Failed to ping master at 127.7.132.190:44921: Client connection negotiation failed: client connection to 127.7.132.190:44921: connect: Connection refused (error 111)
I20260812 02:39:48.211771 26364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:48.211481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:44921 (local address 127.7.132.131:50285)
0812 02:39:48.211604 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0812 02:39:48.211673 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:44921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260812 02:39:48.212544 26364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:48.212236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:44921 (local address 127.7.132.131:51507)
0812 02:39:48.212345 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0812 02:39:48.212412 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:44921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260812 02:39:48.212785 26361 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 02:39:48.274161 26365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:48.273187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:44921 (local address 127.7.132.129:59339)
0812 02:39:48.273752 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0812 02:39:48.273875 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:44921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":278,"threads_started":1}
I20260812 02:39:48.748797 26367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:48.748045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:44921 (local address 127.7.132.130:42385)
0812 02:39:48.748472 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0812 02:39:48.748580 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:44921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":208,"threads_started":1}
I20260812 02:39:49.214236 26368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:49.213361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:44921 (local address 127.7.132.131:47659)
0812 02:39:49.213817 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0812 02:39:49.213924 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:44921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":229,"threads_started":1}
I20260812 02:39:49.275804 26369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:49.275024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:44921 (local address 127.7.132.129:53931)
0812 02:39:49.275543 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0812 02:39:49.275633 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:44921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":272,"threads_started":1}
I20260812 02:39:49.750560 26370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:49.749682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:44921 (local address 127.7.132.130:52539)
0812 02:39:49.750238 (+ 556us) negotiation.cc:107] Waiting for socket to connect
0812 02:39:49.750327 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:44921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":254,"threads_started":1}
I20260812 02:39:50.215852 26371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:50.215152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:44921 (local address 127.7.132.131:55267)
0812 02:39:50.215607 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0812 02:39:50.215686 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:44921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":234,"threads_started":1}
I20260812 02:39:50.277486 26372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:50.276673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:44921 (local address 127.7.132.129:47675)
0812 02:39:50.277207 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0812 02:39:50.277304 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:44921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":287,"threads_started":1}
I20260812 02:39:50.752178 26373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:50.751461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:44921 (local address 127.7.132.130:53403)
0812 02:39:50.751914 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0812 02:39:50.751999 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:44921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":238,"threads_started":1}
I20260812 02:39:51.217512 26374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:51.216789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:44921 (local address 127.7.132.131:53817)
0812 02:39:51.217238 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0812 02:39:51.217321 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:44921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":229,"threads_started":1}
I20260812 02:39:51.279172 26376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:51.278414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:44921 (local address 127.7.132.129:38091)
0812 02:39:51.278865 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0812 02:39:51.278952 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:44921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":234,"threads_started":1}
I20260812 02:39:51.753799 26377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:51.753084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:44921 (local address 127.7.132.130:54837)
0812 02:39:51.753521 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0812 02:39:51.753605 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:44921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":217,"threads_started":1}
I20260812 02:39:52.219555 26378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:52.218571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:44921 (local address 127.7.132.131:60185)
0812 02:39:52.219158 (+ 587us) negotiation.cc:107] Waiting for socket to connect
0812 02:39:52.219271 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:44921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":274,"threads_started":1}
I20260812 02:39:52.280970 26379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:52.280154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:44921 (local address 127.7.132.129:38303)
0812 02:39:52.280704 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0812 02:39:52.280792 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:44921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":274,"threads_started":1}
I20260812 02:39:52.755550 26380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:52.754807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:44921 (local address 127.7.132.130:51233)
0812 02:39:52.755282 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0812 02:39:52.755368 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:44921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":247,"threads_started":1}
I20260812 02:39:53.221436 26381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:53.220447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:44921 (local address 127.7.132.131:35763)
0812 02:39:53.221097 (+ 650us) negotiation.cc:107] Waiting for socket to connect
0812 02:39:53.221202 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:44921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":530,"thread_start_us":347,"threads_started":1}
I20260812 02:39:53.282680 26382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:53.281913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:44921 (local address 127.7.132.129:38995)
0812 02:39:53.282416 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0812 02:39:53.282508 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:44921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":283,"threads_started":1}
I20260812 02:39:53.757261 26383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:53.756437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:44921 (local address 127.7.132.130:50859)
0812 02:39:53.756996 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0812 02:39:53.757097 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:44921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":335,"threads_started":1}
I20260812 02:39:54.223201 26384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:54.222426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:44921 (local address 127.7.132.131:55633)
0812 02:39:54.222949 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0812 02:39:54.223034 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:44921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":415,"thread_start_us":306,"threads_started":1}
I20260812 02:39:54.284356 26386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:54.283575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:44921 (local address 127.7.132.129:52413)
0812 02:39:54.284050 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0812 02:39:54.284135 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:44921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":209,"threads_started":1}
I20260812 02:39:54.759030 26387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:54.758230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:44921 (local address 127.7.132.130:50491)
0812 02:39:54.758729 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0812 02:39:54.758810 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:44921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":226,"threads_started":1}
I20260812 02:39:55.224949 26388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:55.224153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:44921 (local address 127.7.132.131:51543)
0812 02:39:55.224654 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0812 02:39:55.224734 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:44921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":226,"threads_started":1}
I20260812 02:39:55.286031 26389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:55.285292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:44921 (local address 127.7.132.129:52063)
0812 02:39:55.285763 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0812 02:39:55.285854 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:44921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":246,"threads_started":1}
I20260812 02:39:55.760808 26390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:55.760040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:44921 (local address 127.7.132.130:38943)
0812 02:39:55.760538 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0812 02:39:55.760632 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:44921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":268,"threads_started":1}
I20260812 02:39:56.226786 26391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:56.225962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:44921 (local address 127.7.132.131:52561)
0812 02:39:56.226499 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0812 02:39:56.226594 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:44921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":320,"threads_started":1}
I20260812 02:39:56.287943 26392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:56.287094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:44921 (local address 127.7.132.129:47159)
0812 02:39:56.287633 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0812 02:39:56.287719 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:44921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":274,"threads_started":1}
I20260812 02:39:56.762727 26393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:56.761830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:44921 (local address 127.7.132.130:54205)
0812 02:39:56.762447 (+ 617us) negotiation.cc:107] Waiting for socket to connect
0812 02:39:56.762541 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:44921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":506,"thread_start_us":366,"threads_started":1}
I20260812 02:39:57.228466 26394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:57.227709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:44921 (local address 127.7.132.131:48245)
0812 02:39:57.228204 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0812 02:39:57.228286 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:44921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":236,"threads_started":1}
I20260812 02:39:57.289613 26395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:57.288881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:44921 (local address 127.7.132.129:48243)
0812 02:39:57.289355 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0812 02:39:57.289440 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:44921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":255,"threads_started":1}
I20260812 02:39:57.764433 26397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:57.763712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:44921 (local address 127.7.132.130:46311)
0812 02:39:57.764185 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0812 02:39:57.764270 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:44921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":252,"threads_started":1}
I20260812 02:39:58.230314 26399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:58.229545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:44921 (local address 127.7.132.131:55879)
0812 02:39:58.229989 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0812 02:39:58.230149 (+ 160us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:44921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":244,"threads_started":1}
I20260812 02:39:58.291433 26400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:58.290611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:44921 (local address 127.7.132.129:55301)
0812 02:39:58.291140 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0812 02:39:58.291236 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:44921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":293,"threads_started":1}
I20260812 02:39:58.766131 26401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:58.765344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:44921 (local address 127.7.132.130:50541)
0812 02:39:58.765802 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0812 02:39:58.765894 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:44921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":232,"threads_started":1}
I20260812 02:39:59.232196 26402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:59.231352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:44921 (local address 127.7.132.131:48531)
0812 02:39:59.231854 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0812 02:39:59.231991 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:44921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":260,"threads_started":1}
I20260812 02:39:59.293242 26404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:59.292523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:44921 (local address 127.7.132.129:55415)
0812 02:39:59.292966 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0812 02:39:59.293076 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:44921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":225,"threads_started":1}
I20260812 02:39:59.767874 26405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:39:59.767171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:44921 (local address 127.7.132.130:53881)
0812 02:39:59.767628 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0812 02:39:59.767715 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:44921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":225,"threads_started":1}
I20260812 02:40:00.233956 26407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:40:00.233248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:44921 (local address 127.7.132.131:48335)
0812 02:40:00.233697 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0812 02:40:00.233783 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:44921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":226,"threads_started":1}
I20260812 02:40:00.295255 26408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:40:00.294370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:44921 (local address 127.7.132.129:51083)
0812 02:40:00.294932 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0812 02:40:00.295049 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:44921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":265,"threads_started":1}
I20260812 02:40:00.769598 26409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:40:00.768791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:44921 (local address 127.7.132.130:44183)
0812 02:40:00.769297 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0812 02:40:00.769397 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:44921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":252,"threads_started":1}
I20260812 02:40:01.235734 26410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:40:01.234972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:44921 (local address 127.7.132.131:56895)
0812 02:40:01.235444 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0812 02:40:01.235539 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:44921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":238,"threads_started":1}
I20260812 02:40:01.297135 26412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:40:01.296348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:44921 (local address 127.7.132.129:42877)
0812 02:40:01.296830 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0812 02:40:01.296928 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:44921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":246,"threads_started":1}
I20260812 02:40:01.771339 26413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:40:01.770549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:44921 (local address 127.7.132.130:42487)
0812 02:40:01.771054 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0812 02:40:01.771153 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:44921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":286,"threads_started":1}
I20260812 02:40:02.237349 26416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:40:02.236641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:44921 (local address 127.7.132.131:46773)
0812 02:40:02.237084 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0812 02:40:02.237167 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:44921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":234,"threads_started":1}
I20260812 02:40:02.299288 26417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:40:02.298389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:44921 (local address 127.7.132.129:43753)
0812 02:40:02.298881 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0812 02:40:02.299035 (+ 154us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:44921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":241,"threads_started":1}
I20260812 02:40:02.773347 26418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:40:02.772558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:44921 (local address 127.7.132.130:34901)
0812 02:40:02.773062 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0812 02:40:02.773158 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:44921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":254,"threads_started":1}
I20260812 02:40:03.209049 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 25869
I20260812 02:40:03.228737 26419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:40:03.210347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41376 (local address 127.7.132.129:41103)
0812 02:40:03.211029 (+ 682us) server_negotiation.cc:206] Beginning negotiation
0812 02:40:03.211052 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0812 02:40:03.211121 (+ 69us) server_negotiation.cc:407] Connection header received
0812 02:40:03.212833 (+ 1712us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:40:03.212845 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:40:03.213309 (+ 464us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:40:03.213690 (+ 381us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:40:03.215422 (+ 1732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:40:03.216764 (+ 1342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:40:03.218040 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:40:03.218804 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:40:03.219028 (+ 224us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:40:03.221844 (+ 2816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:40:03.221853 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:40:03.221869 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:40:03.221934 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:40:03.225052 (+ 3118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:40:03.225839 (+ 787us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:40:03.225847 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:40:03.225854 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:40:03.226037 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:40:03.226527 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:40:03.226532 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:40:03.226536 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:40:03.227104 (+ 568us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:40:03.227449 (+ 345us) server_negotiation.cc:1035] Waiting for connection context
0812 02:40:03.227974 (+ 525us) server_negotiation.cc:299] Negotiation successful
0812 02:40:03.228437 (+ 463us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":420,"thread_start_us":270,"threads_started":1}
I20260812 02:40:03.230324 26048 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:40:03.239706 26421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:40:03.238706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:44921 (local address 127.7.132.131:34891)
0812 02:40:03.239377 (+ 671us) negotiation.cc:107] Waiting for socket to connect
0812 02:40:03.239471 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:44921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":505,"thread_start_us":317,"threads_started":1}
I20260812 02:40:03.324865 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 25869
I20260812 02:40:03.351536 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 26056
I20260812 02:40:03.369674 26426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:40:03.352567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56490 (local address 127.7.132.130:41397)
0812 02:40:03.353107 (+ 540us) server_negotiation.cc:206] Beginning negotiation
0812 02:40:03.353122 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 02:40:03.353164 (+ 42us) server_negotiation.cc:407] Connection header received
0812 02:40:03.354886 (+ 1722us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:40:03.354903 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:40:03.355367 (+ 464us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:40:03.355693 (+ 326us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:40:03.357209 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:40:03.358593 (+ 1384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:40:03.359837 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:40:03.360506 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:40:03.360639 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:40:03.363347 (+ 2708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:40:03.363354 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:40:03.363368 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:40:03.363413 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:40:03.366547 (+ 3134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:40:03.367223 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:40:03.367231 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:40:03.367237 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:40:03.367424 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:40:03.367769 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:40:03.367773 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:40:03.367775 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:40:03.368321 (+ 546us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:40:03.368566 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0812 02:40:03.368993 (+ 427us) server_negotiation.cc:299] Negotiation successful
0812 02:40:03.369346 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":379,"thread_start_us":533,"threads_started":1}
I20260812 02:40:03.371305 26197 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:40:03.456851 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 26056
I20260812 02:40:03.480964 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 26206
I20260812 02:40:03.500615 26430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:40:03.482053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55952 (local address 127.7.132.131:46073)
0812 02:40:03.482758 (+ 705us) server_negotiation.cc:206] Beginning negotiation
0812 02:40:03.482773 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 02:40:03.482817 (+ 44us) server_negotiation.cc:407] Connection header received
0812 02:40:03.484503 (+ 1686us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:40:03.484513 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:40:03.484986 (+ 473us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:40:03.485362 (+ 376us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:40:03.487162 (+ 1800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:40:03.488454 (+ 1292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:40:03.489830 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:40:03.490680 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:40:03.490905 (+ 225us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:40:03.493730 (+ 2825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:40:03.493740 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:40:03.493757 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:40:03.493799 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:40:03.496837 (+ 3038us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:40:03.497635 (+ 798us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:40:03.497643 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:40:03.497650 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:40:03.497827 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:40:03.498330 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:40:03.498339 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:40:03.498341 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:40:03.498979 (+ 638us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:40:03.499309 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0812 02:40:03.499828 (+ 519us) server_negotiation.cc:299] Negotiation successful
0812 02:40:03.500291 (+ 463us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":542,"thread_start_us":297,"threads_started":1}
I20260812 02:40:03.502230 26355 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:40:03.587729 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 26206
2026-08-12T02:40:03Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (24923 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 02:40:03 dist-test-slave-mn4r krb5kdc[26436](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 02:40:03 dist-test-slave-mn4r krb5kdc[26436](info): set up 2 sockets
Aug 12 02:40:03 dist-test-slave-mn4r krb5kdc[26436](info): commencing operation
krb5kdc: starting...
W20260812 02:40:04.814960 7698 mini_kdc.cc:121] Time spent starting KDC: real 1.182s user 0.004s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/krb5kdc/test-user.keytab.
Aug 12 02:40:05 dist-test-slave-mn4r krb5kdc[26436](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502405, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T02:40:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T02:40:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.132.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
I20260812 02:40:05.114202 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:42683
--webserver_interface=127.7.132.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:44005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.132.190:42683
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:40:05.471345 26450 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:40:05.471658 26450 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:40:05.471835 26450 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:40:05.479300 26450 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260812 02:40:05.479400 26450 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260812 02:40:05.479470 26450 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260812 02:40:05.479552 26450 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:40:05.479645 26450 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:40:05.479763 26450 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:40:05.479835 26450 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:40:05.479921 26450 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:40:05.490157 26450 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:44005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.7.132.190:42683
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:42683
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.132.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:40:05.491607 26450 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:40:05.493054 26450 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:40:05.505939 26458 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:40:05.505981 26459 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:40:05.505867 26461 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:40:05.506646 26450 server_base.cc:1034] running on GCE node
I20260812 02:40:05.507539 26450 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:40:05.509084 26450 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:40:05.510367 26450 hybrid_clock.cc:648] HybridClock initialized: now 1786502405510290 us; error 41 us; skew 500 ppm
I20260812 02:40:05.510802 26450 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:40:05 dist-test-slave-mn4r krb5kdc[26436](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502405, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:40:05.519709 26450 init.cc:377] Logged in from keytab as kudu/127.7.132.190@KRBTEST.COM (short username kudu)
I20260812 02:40:05.521307 26450 webserver.cc:460] Webserver started at http://127.7.132.190:41403/ using document root <none> and password file <none>
I20260812 02:40:05.522051 26450 fs_manager.cc:359] Metadata directory not provided
I20260812 02:40:05.522224 26450 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:40:05.522571 26450 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:40:05.525650 26450 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/master-0/data/instance:
uuid: "0fad46d3a46c4597903b423dfd124d70"
format_stamp: "Formatted at 2026-08-12 02:40:05 on dist-test-slave-mn4r"
server_key: "b731db17ef27b79895faecff3c1d5c27"
server_key_iv: "ed35bab47187d559b4b58d26a970b636"
server_key_version: "encryptionkey@0"
I20260812 02:40:05.526465 26450 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/master-0/wal/instance:
uuid: "0fad46d3a46c4597903b423dfd124d70"
format_stamp: "Formatted at 2026-08-12 02:40:05 on dist-test-slave-mn4r"
server_key: "b731db17ef27b79895faecff3c1d5c27"
server_key_iv: "ed35bab47187d559b4b58d26a970b636"
server_key_version: "encryptionkey@0"
I20260812 02:40:05.531605 26450 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260812 02:40:05.535144 26468 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:40:05.536448 26450 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 02:40:05.536634 26450 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/master-0/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/master-0/wal
uuid: "0fad46d3a46c4597903b423dfd124d70"
format_stamp: "Formatted at 2026-08-12 02:40:05 on dist-test-slave-mn4r"
server_key: "b731db17ef27b79895faecff3c1d5c27"
server_key_iv: "ed35bab47187d559b4b58d26a970b636"
server_key_version: "encryptionkey@0"
I20260812 02:40:05.536856 26450 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:40:05.559574 26450 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:40:05.564560 26450 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:40:05.564791 26450 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:40:05.583192 26450 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.190:42683
I20260812 02:40:05.583237 26526 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.190:42683 every 8 connection(s)
I20260812 02:40:05.584563 26450 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
I20260812 02:40:05.588606 26527 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:40:05.592991 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 26450
I20260812 02:40:05.593344 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/master-0/wal/instance
I20260812 02:40:05.594023 7698 external_mini_cluster.cc:1428] Setting key 9d1bf13dc50d9db2bfd0c6d51637760d
I20260812 02:40:05.601697 26527 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0fad46d3a46c4597903b423dfd124d70: Bootstrap starting.
I20260812 02:40:05.605659 26527 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0fad46d3a46c4597903b423dfd124d70: Neither blocks nor log segments found. Creating new log.
I20260812 02:40:05.607106 26527 log.cc:824] T 00000000000000000000000000000000 P 0fad46d3a46c4597903b423dfd124d70: Log is configured to *not* fsync() on all Append() calls
I20260812 02:40:05.610262 26527 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0fad46d3a46c4597903b423dfd124d70: No bootstrap required, opened a new log
Aug 12 02:40:05 dist-test-slave-mn4r krb5kdc[26436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502405, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 02:40:05.620905 26531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:40:05.596238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59420 (local address 127.7.132.190:42683)
0812 02:40:05.597102 (+ 864us) server_negotiation.cc:206] Beginning negotiation
0812 02:40:05.597119 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 02:40:05.597176 (+ 57us) server_negotiation.cc:407] Connection header received
0812 02:40:05.599844 (+ 2668us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:40:05.599875 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:40:05.600404 (+ 529us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:40:05.600862 (+ 458us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:40:05.602351 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:40:05.604431 (+ 2080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:40:05.605853 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:40:05.607020 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:40:05.607247 (+ 227us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:40:05.611665 (+ 4418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:40:05.611677 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:40:05.611698 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:40:05.611750 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:40:05.616025 (+ 4275us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:40:05.616794 (+ 769us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:40:05.616802 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:40:05.616811 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:40:05.617053 (+ 242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:40:05.617419 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:40:05.617425 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:40:05.617428 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:40:05.618228 (+ 800us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:40:05.618510 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0812 02:40:05.618957 (+ 447us) server_negotiation.cc:299] Negotiation successful
0812 02:40:05.619463 (+ 506us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":588,"thread_start_us":354,"threads_started":1}
I20260812 02:40:05.621476 26527 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0fad46d3a46c4597903b423dfd124d70 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fad46d3a46c4597903b423dfd124d70" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 42683 } }
I20260812 02:40:05.621852 26527 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0fad46d3a46c4597903b423dfd124d70 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:40:05.621989 26527 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0fad46d3a46c4597903b423dfd124d70 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fad46d3a46c4597903b423dfd124d70, State: Initialized, Role: FOLLOWER
I20260812 02:40:05.622725 26527 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0fad46d3a46c4597903b423dfd124d70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fad46d3a46c4597903b423dfd124d70" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 42683 } }
I20260812 02:40:05.623026 26527 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0fad46d3a46c4597903b423dfd124d70 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 02:40:05.623181 26527 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0fad46d3a46c4597903b423dfd124d70 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 02:40:05.623377 26527 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0fad46d3a46c4597903b423dfd124d70 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:40:05.625756 26527 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0fad46d3a46c4597903b423dfd124d70 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fad46d3a46c4597903b423dfd124d70" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 42683 } }
I20260812 02:40:05.626345 26527 leader_election.cc:302] T 00000000000000000000000000000000 P 0fad46d3a46c4597903b423dfd124d70 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0fad46d3a46c4597903b423dfd124d70; no voters:
I20260812 02:40:05.626932 26527 leader_election.cc:288] T 00000000000000000000000000000000 P 0fad46d3a46c4597903b423dfd124d70 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 02:40:05.627104 26533 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0fad46d3a46c4597903b423dfd124d70 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:40:05.627873 26533 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0fad46d3a46c4597903b423dfd124d70 [term 1 LEADER]: Becoming Leader. State: Replica: 0fad46d3a46c4597903b423dfd124d70, State: Running, Role: LEADER
I20260812 02:40:05.628414 26533 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0fad46d3a46c4597903b423dfd124d70 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fad46d3a46c4597903b423dfd124d70" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 42683 } }
I20260812 02:40:05.629093 26527 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0fad46d3a46c4597903b423dfd124d70 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:40:05.631536 26535 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0fad46d3a46c4597903b423dfd124d70 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0fad46d3a46c4597903b423dfd124d70. Latest consensus state: current_term: 1 leader_uuid: "0fad46d3a46c4597903b423dfd124d70" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fad46d3a46c4597903b423dfd124d70" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 42683 } } }
I20260812 02:40:05.631975 26535 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0fad46d3a46c4597903b423dfd124d70 [sys.catalog]: This master's current role is: LEADER
I20260812 02:40:05.634298 26534 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0fad46d3a46c4597903b423dfd124d70 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0fad46d3a46c4597903b423dfd124d70" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fad46d3a46c4597903b423dfd124d70" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 42683 } } }
I20260812 02:40:05.634673 26534 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0fad46d3a46c4597903b423dfd124d70 [sys.catalog]: This master's current role is: LEADER
I20260812 02:40:05.635236 26542 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:40:05.639835 26542 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 02:40:05.649299 26542 catalog_manager.cc:1347] Generated new cluster ID: 40c1dacf063b4f2d90266628e0db9540
I20260812 02:40:05.649446 26542 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 02:40:05.663327 26542 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 02:40:05.664610 26542 catalog_manager.cc:1504] Loading token signing keys...
I20260812 02:40:05.678819 26542 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0fad46d3a46c4597903b423dfd124d70: Generated new TSK 0
I20260812 02:40:05.679736 26542 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.132.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.129.keytab.
Entry for principal kudu/127.7.132.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.129.keytab.
I20260812 02:40:05.755201 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.129:0
--local_ip_for_outbound_sockets=127.7.132.129
--webserver_interface=127.7.132.129
--webserver_port=0
--tserver_master_addrs=127.7.132.190:42683
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.129.keytab
--principal=kudu/127.7.132.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:44005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:40:06.102982 26558 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:40:06.103263 26558 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:40:06.103358 26558 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:40:06.110687 26558 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:40:06.110795 26558 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:40:06.110908 26558 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.129
I20260812 02:40:06.120656 26558 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:44005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.129.keytab
--principal=kudu/127.7.132.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.132.129
--webserver_port=0
--tserver_master_addrs=127.7.132.190:42683
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.129
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:40:06.121955 26558 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:40:06.123383 26558 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:40:06.136955 26565 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:40:06.136996 26566 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:40:06.137036 26568 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:40:06.138288 26558 server_base.cc:1034] running on GCE node
I20260812 02:40:06.138888 26558 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:40:06.139835 26558 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:40:06.141073 26558 hybrid_clock.cc:648] HybridClock initialized: now 1786502406141016 us; error 50 us; skew 500 ppm
I20260812 02:40:06.141434 26558 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:40:06 dist-test-slave-mn4r krb5kdc[26436](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502406, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:40:06.144819 26558 init.cc:377] Logged in from keytab as kudu/127.7.132.129@KRBTEST.COM (short username kudu)
I20260812 02:40:06.146368 26558 webserver.cc:460] Webserver started at http://127.7.132.129:33519/ using document root <none> and password file <none>
I20260812 02:40:06.147017 26558 fs_manager.cc:359] Metadata directory not provided
I20260812 02:40:06.147123 26558 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:40:06.147392 26558 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:40:06.150277 26558 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-0/data/instance:
uuid: "cabf179ab4374509b0163a571e20f483"
format_stamp: "Formatted at 2026-08-12 02:40:06 on dist-test-slave-mn4r"
server_key: "dfe9a185d5e7020d0b76ae28c460f70b"
server_key_iv: "77e6c572a50b660125dad0e8fca42854"
server_key_version: "encryptionkey@0"
I20260812 02:40:06.150993 26558 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-0/wal/instance:
uuid: "cabf179ab4374509b0163a571e20f483"
format_stamp: "Formatted at 2026-08-12 02:40:06 on dist-test-slave-mn4r"
server_key: "dfe9a185d5e7020d0b76ae28c460f70b"
server_key_iv: "77e6c572a50b660125dad0e8fca42854"
server_key_version: "encryptionkey@0"
I20260812 02:40:06.156229 26558 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260812 02:40:06.159723 26579 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:40:06.161059 26558 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 02:40:06.161231 26558 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-0/wal
uuid: "cabf179ab4374509b0163a571e20f483"
format_stamp: "Formatted at 2026-08-12 02:40:06 on dist-test-slave-mn4r"
server_key: "dfe9a185d5e7020d0b76ae28c460f70b"
server_key_iv: "77e6c572a50b660125dad0e8fca42854"
server_key_version: "encryptionkey@0"
I20260812 02:40:06.161410 26558 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:40:06.178733 26558 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:40:06.183279 26558 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:40:06.183573 26558 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:40:06.184610 26558 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:40:06.185953 26558 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:40:06.186043 26558 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:40:06.186224 26558 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:40:06.186276 26558 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:40:06.219838 26558 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.129:41881
I20260812 02:40:06.219955 26713 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.129:41881 every 8 connection(s)
I20260812 02:40:06.221168 26558 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-0/data/info.pb
I20260812 02:40:06.225131 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 26558
I20260812 02:40:06.225363 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-0/wal/instance
I20260812 02:40:06.226183 7698 external_mini_cluster.cc:1428] Setting key f5c38bafffcd2827215c8402ee4add21
Aug 12 02:40:06 dist-test-slave-mn4r krb5kdc[26436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502406, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.129@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 02:40:06.244539 26716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:40:06.224053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:42107 (local address 127.7.132.190:42683)
0812 02:40:06.224513 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0812 02:40:06.224518 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:40:06.225769 (+ 1251us) server_negotiation.cc:407] Connection header received
0812 02:40:06.228143 (+ 2374us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:40:06.228147 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:40:06.228212 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:40:06.228334 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:40:06.230057 (+ 1723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:40:06.231282 (+ 1225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:40:06.233253 (+ 1971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:40:06.233947 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:40:06.234141 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:40:06.237838 (+ 3697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:40:06.237847 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:40:06.237855 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:40:06.237903 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:40:06.241064 (+ 3161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:40:06.242038 (+ 974us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:40:06.242042 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:40:06.242067 (+ 25us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:40:06.242238 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:40:06.242749 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:40:06.242752 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:40:06.242754 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:40:06.243018 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:40:06.243144 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0812 02:40:06.244106 (+ 962us) server_negotiation.cc:299] Negotiation successful
0812 02:40:06.244283 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":347,"thread_start_us":225,"threads_started":1}
I20260812 02:40:06.246248 26718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:40:06.224637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:42683 (local address 127.7.132.129:42107)
0812 02:40:06.225554 (+ 917us) negotiation.cc:107] Waiting for socket to connect
0812 02:40:06.225630 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0812 02:40:06.227813 (+ 2183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:40:06.228573 (+ 760us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:40:06.228582 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:40:06.229212 (+ 630us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:40:06.229857 (+ 645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:40:06.229878 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:40:06.231488 (+ 1610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:40:06.231492 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:40:06.233111 (+ 1619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:40:06.233128 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:40:06.234224 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:40:06.234230 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:40:06.234446 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:40:06.235157 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:40:06.235179 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:40:06.237600 (+ 2421us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:40:06.241201 (+ 3601us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:40:06.241212 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:40:06.241230 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:40:06.241926 (+ 696us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:40:06.242349 (+ 423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:40:06.242357 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:40:06.242364 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:40:06.242639 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:40:06.243175 (+ 536us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:40:06.243186 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:40:06.243673 (+ 487us) client_negotiation.cc:769] Sending connection context
0812 02:40:06.244008 (+ 335us) client_negotiation.cc:240] Negotiation successful
0812 02:40:06.244657 (+ 649us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":610,"thread_start_us":394,"threads_started":1}
I20260812 02:40:06.248010 26714 heartbeater.cc:347] Connected to a master server at 127.7.132.190:42683
I20260812 02:40:06.248351 26714 heartbeater.cc:464] Registering TS with master...
I20260812 02:40:06.249133 26714 heartbeater.cc:511] Master 127.7.132.190:42683 requested a full tablet report, sending...
I20260812 02:40:06.251446 26488 ts_manager.cc:194] Registered new tserver with Master: cabf179ab4374509b0163a571e20f483 (127.7.132.129:41881)
I20260812 02:40:06.252940 26488 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.129@KRBTEST.COM'} at 127.7.132.129:42107
WARNING: no policy specified for kudu/127.7.132.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.130.keytab.
Entry for principal kudu/127.7.132.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.130.keytab.
I20260812 02:40:06.295504 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.130:0
--local_ip_for_outbound_sockets=127.7.132.130
--webserver_interface=127.7.132.130
--webserver_port=0
--tserver_master_addrs=127.7.132.190:42683
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.130.keytab
--principal=kudu/127.7.132.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:44005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:40:06.649890 26721 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:40:06.650295 26721 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:40:06.650461 26721 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:40:06.658329 26721 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:40:06.658478 26721 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:40:06.658643 26721 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.130
I20260812 02:40:06.668957 26721 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:44005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.130.keytab
--principal=kudu/127.7.132.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.132.130
--webserver_port=0
--tserver_master_addrs=127.7.132.190:42683
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.130
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:40:06.670346 26721 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:40:06.671729 26721 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:40:06.685159 26730 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:40:06.685199 26734 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:40:06.685249 26731 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:40:06.686044 26721 server_base.cc:1034] running on GCE node
I20260812 02:40:06.686743 26721 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:40:06.687700 26721 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:40:06.688961 26721 hybrid_clock.cc:648] HybridClock initialized: now 1786502406688828 us; error 116 us; skew 500 ppm
I20260812 02:40:06.689328 26721 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:40:06 dist-test-slave-mn4r krb5kdc[26436](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502406, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:40:06.692767 26721 init.cc:377] Logged in from keytab as kudu/127.7.132.130@KRBTEST.COM (short username kudu)
I20260812 02:40:06.694474 26721 webserver.cc:460] Webserver started at http://127.7.132.130:36611/ using document root <none> and password file <none>
I20260812 02:40:06.695200 26721 fs_manager.cc:359] Metadata directory not provided
I20260812 02:40:06.695346 26721 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:40:06.695680 26721 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:40:06.698783 26721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-1/data/instance:
uuid: "c2298009914a4733a6b0604dd0843833"
format_stamp: "Formatted at 2026-08-12 02:40:06 on dist-test-slave-mn4r"
server_key: "1b234fb35756c4ca2b9a5ad2ac41f3fd"
server_key_iv: "0df15e5f009641f6ebcb4ec45d2108ba"
server_key_version: "encryptionkey@0"
I20260812 02:40:06.699582 26721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-1/wal/instance:
uuid: "c2298009914a4733a6b0604dd0843833"
format_stamp: "Formatted at 2026-08-12 02:40:06 on dist-test-slave-mn4r"
server_key: "1b234fb35756c4ca2b9a5ad2ac41f3fd"
server_key_iv: "0df15e5f009641f6ebcb4ec45d2108ba"
server_key_version: "encryptionkey@0"
I20260812 02:40:06.705194 26721 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260812 02:40:06.708803 26742 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:40:06.710213 26721 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260812 02:40:06.710386 26721 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-1/wal
uuid: "c2298009914a4733a6b0604dd0843833"
format_stamp: "Formatted at 2026-08-12 02:40:06 on dist-test-slave-mn4r"
server_key: "1b234fb35756c4ca2b9a5ad2ac41f3fd"
server_key_iv: "0df15e5f009641f6ebcb4ec45d2108ba"
server_key_version: "encryptionkey@0"
I20260812 02:40:06.710592 26721 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:40:06.730975 26721 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:40:06.735322 26721 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:40:06.735615 26721 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:40:06.736574 26721 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:40:06.737906 26721 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:40:06.738006 26721 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:40:06.738175 26721 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:40:06.738247 26721 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:40:06.771710 26721 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.130:37905
I20260812 02:40:06.771763 26870 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.130:37905 every 8 connection(s)
I20260812 02:40:06.773038 26721 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-1/data/info.pb
I20260812 02:40:06.774765 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 26721
I20260812 02:40:06.774969 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-1/wal/instance
I20260812 02:40:06.775454 7698 external_mini_cluster.cc:1428] Setting key 310965997d7ceee001b070f8866bd9d7
Aug 12 02:40:06 dist-test-slave-mn4r krb5kdc[26436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502406, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.130@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 02:40:06.793449 26873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:40:06.775704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:50441 (local address 127.7.132.190:42683)
0812 02:40:06.776248 (+ 544us) server_negotiation.cc:206] Beginning negotiation
0812 02:40:06.776255 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:40:06.777298 (+ 1043us) server_negotiation.cc:407] Connection header received
0812 02:40:06.779259 (+ 1961us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:40:06.779264 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:40:06.779353 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:40:06.779468 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:40:06.780946 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:40:06.781693 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:40:06.783315 (+ 1622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:40:06.783756 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:40:06.783885 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:40:06.787314 (+ 3429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:40:06.787319 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:40:06.787322 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:40:06.787353 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:40:06.790474 (+ 3121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:40:06.791201 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:40:06.791206 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:40:06.791210 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:40:06.791347 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:40:06.791772 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:40:06.791776 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:40:06.791778 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:40:06.792024 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:40:06.792176 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0812 02:40:06.792833 (+ 657us) server_negotiation.cc:299] Negotiation successful
0812 02:40:06.793054 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":401,"thread_start_us":269,"threads_started":1}
I20260812 02:40:06.794534 26874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:40:06.776302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:42683 (local address 127.7.132.130:50441)
0812 02:40:06.777134 (+ 832us) negotiation.cc:107] Waiting for socket to connect
0812 02:40:06.777189 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0812 02:40:06.778993 (+ 1804us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:40:06.779713 (+ 720us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:40:06.779721 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:40:06.780224 (+ 503us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:40:06.780781 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:40:06.780800 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:40:06.781872 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:40:06.781876 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:40:06.783174 (+ 1298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:40:06.783187 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:40:06.783984 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:40:06.783991 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:40:06.784230 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:40:06.784901 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:40:06.784927 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:40:06.787091 (+ 2164us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:40:06.790636 (+ 3545us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:40:06.790645 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:40:06.790661 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:40:06.791076 (+ 415us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:40:06.791455 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:40:06.791461 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:40:06.791464 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:40:06.791650 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:40:06.792149 (+ 499us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:40:06.792157 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:40:06.792483 (+ 326us) client_negotiation.cc:769] Sending connection context
0812 02:40:06.792742 (+ 259us) client_negotiation.cc:240] Negotiation successful
0812 02:40:06.793286 (+ 544us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":546,"thread_start_us":339,"threads_started":1}
I20260812 02:40:06.796175 26871 heartbeater.cc:347] Connected to a master server at 127.7.132.190:42683
I20260812 02:40:06.796484 26871 heartbeater.cc:464] Registering TS with master...
I20260812 02:40:06.797228 26871 heartbeater.cc:511] Master 127.7.132.190:42683 requested a full tablet report, sending...
I20260812 02:40:06.798741 26488 ts_manager.cc:194] Registered new tserver with Master: c2298009914a4733a6b0604dd0843833 (127.7.132.130:37905)
I20260812 02:40:06.799569 26488 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.130@KRBTEST.COM'} at 127.7.132.130:50441
WARNING: no policy specified for kudu/127.7.132.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.131.keytab.
Entry for principal kudu/127.7.132.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.131.keytab.
I20260812 02:40:06.840941 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.131:0
--local_ip_for_outbound_sockets=127.7.132.131
--webserver_interface=127.7.132.131
--webserver_port=0
--tserver_master_addrs=127.7.132.190:42683
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.131.keytab
--principal=kudu/127.7.132.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:44005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:40:07.184175 26877 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:40:07.184473 26877 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:40:07.184568 26877 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:40:07.192075 26877 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:40:07.192193 26877 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:40:07.192306 26877 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.131
I20260812 02:40:07.202405 26877 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:44005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.131.keytab
--principal=kudu/127.7.132.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.132.131
--webserver_port=0
--tserver_master_addrs=127.7.132.190:42683
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.131
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:40:07.203708 26877 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:40:07.205098 26877 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:40:07.218590 26886 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:40:07.218647 26884 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:40:07.218847 26883 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:40:07.222779 26877 server_base.cc:1034] running on GCE node
I20260812 02:40:07.223438 26877 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:40:07.224440 26877 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:40:07.225698 26877 hybrid_clock.cc:648] HybridClock initialized: now 1786502407225650 us; error 52 us; skew 500 ppm
I20260812 02:40:07.226042 26877 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:40:07 dist-test-slave-mn4r krb5kdc[26436](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502407, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:40:07.229324 26877 init.cc:377] Logged in from keytab as kudu/127.7.132.131@KRBTEST.COM (short username kudu)
I20260812 02:40:07.230922 26877 webserver.cc:460] Webserver started at http://127.7.132.131:40209/ using document root <none> and password file <none>
I20260812 02:40:07.231643 26877 fs_manager.cc:359] Metadata directory not provided
I20260812 02:40:07.231818 26877 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:40:07.232149 26877 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:40:07.235240 26877 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-2/data/instance:
uuid: "6ae48edd11834d13a65cb21c37afb69a"
format_stamp: "Formatted at 2026-08-12 02:40:07 on dist-test-slave-mn4r"
server_key: "ffd82d769373b97c9960a53afc1bfa1e"
server_key_iv: "1ae78bcf9dea599eaaa8593e58113d0c"
server_key_version: "encryptionkey@0"
I20260812 02:40:07.236038 26877 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-2/wal/instance:
uuid: "6ae48edd11834d13a65cb21c37afb69a"
format_stamp: "Formatted at 2026-08-12 02:40:07 on dist-test-slave-mn4r"
server_key: "ffd82d769373b97c9960a53afc1bfa1e"
server_key_iv: "1ae78bcf9dea599eaaa8593e58113d0c"
server_key_version: "encryptionkey@0"
I20260812 02:40:07.241495 26877 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260812 02:40:07.245033 26893 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:40:07.246574 26877 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 02:40:07.246765 26877 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-2/wal
uuid: "6ae48edd11834d13a65cb21c37afb69a"
format_stamp: "Formatted at 2026-08-12 02:40:07 on dist-test-slave-mn4r"
server_key: "ffd82d769373b97c9960a53afc1bfa1e"
server_key_iv: "1ae78bcf9dea599eaaa8593e58113d0c"
server_key_version: "encryptionkey@0"
I20260812 02:40:07.246970 26877 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:40:07.255980 26714 heartbeater.cc:503] Master 127.7.132.190:42683 was elected leader, sending a full tablet report...
I20260812 02:40:07.262549 26877 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:40:07.267217 26877 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:40:07.267534 26877 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:40:07.268539 26877 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:40:07.269958 26877 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:40:07.270061 26877 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:40:07.270236 26877 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:40:07.270313 26877 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:40:07.305482 26877 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.131:34721
I20260812 02:40:07.305527 27025 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.131:34721 every 8 connection(s)
I20260812 02:40:07.306903 26877 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-2/data/info.pb
I20260812 02:40:07.310565 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 26877
I20260812 02:40:07.310765 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-2/wal/instance
I20260812 02:40:07.311380 7698 external_mini_cluster.cc:1428] Setting key d5f2075cb9599356b34a8f10d631d034
Aug 12 02:40:07 dist-test-slave-mn4r krb5kdc[26436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502407, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.131@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 02:40:07.327100 27029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:40:07.309429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:32913 (local address 127.7.132.190:42683)
0812 02:40:07.309863 (+ 434us) server_negotiation.cc:206] Beginning negotiation
0812 02:40:07.309868 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:40:07.311058 (+ 1190us) server_negotiation.cc:407] Connection header received
0812 02:40:07.312928 (+ 1870us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:40:07.312933 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:40:07.313051 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:40:07.313169 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:40:07.314787 (+ 1618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:40:07.315710 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:40:07.317224 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:40:07.317621 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:40:07.317749 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:40:07.321171 (+ 3422us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:40:07.321176 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:40:07.321179 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:40:07.321211 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:40:07.324058 (+ 2847us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:40:07.324768 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:40:07.324772 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:40:07.324775 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:40:07.324895 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:40:07.325330 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:40:07.325335 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:40:07.325342 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:40:07.325600 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:40:07.325762 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0812 02:40:07.326487 (+ 725us) server_negotiation.cc:299] Negotiation successful
0812 02:40:07.326710 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":200,"threads_started":1}
I20260812 02:40:07.328023 27030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:40:07.309993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:42683 (local address 127.7.132.131:32913)
0812 02:40:07.310903 (+ 910us) negotiation.cc:107] Waiting for socket to connect
0812 02:40:07.310942 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 02:40:07.312666 (+ 1724us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:40:07.313407 (+ 741us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:40:07.313416 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:40:07.313904 (+ 488us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:40:07.314616 (+ 712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:40:07.314643 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:40:07.315855 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:40:07.315861 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:40:07.317079 (+ 1218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:40:07.317091 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:40:07.317848 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:40:07.317855 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:40:07.318143 (+ 288us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:40:07.318821 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:40:07.318843 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:40:07.320947 (+ 2104us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:40:07.324194 (+ 3247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:40:07.324203 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:40:07.324219 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:40:07.324638 (+ 419us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:40:07.325009 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:40:07.325016 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:40:07.325020 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:40:07.325209 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:40:07.325729 (+ 520us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:40:07.325737 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:40:07.326063 (+ 326us) client_negotiation.cc:769] Sending connection context
0812 02:40:07.326380 (+ 317us) client_negotiation.cc:240] Negotiation successful
0812 02:40:07.326912 (+ 532us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":586,"thread_start_us":339,"threads_started":1}
I20260812 02:40:07.329288 27027 heartbeater.cc:347] Connected to a master server at 127.7.132.190:42683
I20260812 02:40:07.329635 27027 heartbeater.cc:464] Registering TS with master...
I20260812 02:40:07.330428 27027 heartbeater.cc:511] Master 127.7.132.190:42683 requested a full tablet report, sending...
I20260812 02:40:07.331868 26488 ts_manager.cc:194] Registered new tserver with Master: 6ae48edd11834d13a65cb21c37afb69a (127.7.132.131:34721)
I20260812 02:40:07.332669 26488 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.131@KRBTEST.COM'} at 127.7.132.131:32913
I20260812 02:40:07.338640 7698 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 02:40:07.357306 27029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:40:07.343225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42014 (local address 127.7.132.190:42683)
0812 02:40:07.343385 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0812 02:40:07.343390 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:40:07.344039 (+ 649us) server_negotiation.cc:407] Connection header received
0812 02:40:07.344222 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:40:07.344227 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:40:07.344302 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:40:07.344403 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:40:07.345850 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:40:07.346633 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:40:07.348154 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:40:07.348586 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:40:07.348724 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:40:07.351774 (+ 3050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:40:07.351780 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:40:07.351785 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:40:07.351827 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:40:07.355060 (+ 3233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:40:07.355680 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:40:07.355683 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:40:07.355686 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:40:07.355800 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:40:07.356115 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:40:07.356118 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:40:07.356120 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:40:07.356428 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:40:07.356624 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0812 02:40:07.356903 (+ 279us) server_negotiation.cc:299] Negotiation successful
0812 02:40:07.357078 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 02:40:07.361469 26488 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42014:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 02:40:07.368527 26488 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 02:40:07.393401 27049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:40:07.382510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.131:34721 (local address 127.0.0.1:38538)
0812 02:40:07.383992 (+ 1482us) negotiation.cc:107] Waiting for socket to connect
0812 02:40:07.384014 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 02:40:07.384145 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:40:07.385342 (+ 1197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:40:07.385350 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:40:07.385381 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:40:07.385755 (+ 374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:40:07.385772 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:40:07.387804 (+ 2032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:40:07.387809 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:40:07.390262 (+ 2453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:40:07.390278 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:40:07.392392 (+ 2114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:40:07.392403 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:40:07.392607 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:40:07.392659 (+ 52us) client_negotiation.cc:769] Sending connection context
0812 02:40:07.392809 (+ 150us) client_negotiation.cc:240] Negotiation successful
0812 02:40:07.393003 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1346,"thread_start_us":1230,"threads_started":1}
I20260812 02:40:07.393401 27050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:40:07.383089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38538 (local address 127.7.132.131:34721)
0812 02:40:07.384834 (+ 1745us) server_negotiation.cc:206] Beginning negotiation
0812 02:40:07.384840 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:40:07.384872 (+ 32us) server_negotiation.cc:407] Connection header received
0812 02:40:07.384950 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:40:07.384956 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:40:07.385139 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:40:07.385335 (+ 196us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:40:07.385977 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:40:07.387630 (+ 1653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:40:07.390445 (+ 2815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:40:07.392164 (+ 1719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:40:07.392308 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:40:07.392546 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0812 02:40:07.392873 (+ 327us) server_negotiation.cc:299] Negotiation successful
0812 02:40:07.393066 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1604,"thread_start_us":1430,"threads_started":1}
I20260812 02:40:07.394799 27043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:40:07.380881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.130:37905 (local address 127.0.0.1:55166)
0812 02:40:07.382444 (+ 1563us) negotiation.cc:107] Waiting for socket to connect
0812 02:40:07.382460 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 02:40:07.382576 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:40:07.387614 (+ 5038us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:40:07.387619 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:40:07.387646 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:40:07.388120 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:40:07.388135 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:40:07.390046 (+ 1911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:40:07.390052 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:40:07.391806 (+ 1754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:40:07.391819 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:40:07.393658 (+ 1839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:40:07.393664 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:40:07.393838 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:40:07.393875 (+ 37us) client_negotiation.cc:769] Sending connection context
0812 02:40:07.393940 (+ 65us) client_negotiation.cc:240] Negotiation successful
0812 02:40:07.394235 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1427,"mutex_wait_us":1,"thread_start_us":1793,"threads_started":1}
I20260812 02:40:07.395241 27048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:40:07.381102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55166 (local address 127.7.132.130:37905)
0812 02:40:07.387072 (+ 5970us) server_negotiation.cc:206] Beginning negotiation
0812 02:40:07.387080 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:40:07.387111 (+ 31us) server_negotiation.cc:407] Connection header received
0812 02:40:07.387208 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:40:07.387215 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:40:07.387420 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:40:07.387629 (+ 209us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:40:07.388382 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:40:07.389892 (+ 1510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:40:07.392000 (+ 2108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:40:07.393424 (+ 1424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:40:07.394385 (+ 961us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:40:07.394601 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0812 02:40:07.394676 (+ 75us) server_negotiation.cc:299] Negotiation successful
0812 02:40:07.394882 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5820,"thread_start_us":5691,"threads_started":1}
I20260812 02:40:07.397445 27042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:40:07.380723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:41881 (local address 127.0.0.1:53776)
0812 02:40:07.381565 (+ 842us) negotiation.cc:107] Waiting for socket to connect
0812 02:40:07.381617 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0812 02:40:07.381828 (+ 211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:40:07.388115 (+ 6287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:40:07.388120 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:40:07.388145 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:40:07.388594 (+ 449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:40:07.388607 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:40:07.390512 (+ 1905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:40:07.390520 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:40:07.392761 (+ 2241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:40:07.392796 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:40:07.396650 (+ 3854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:40:07.396657 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:40:07.396828 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:40:07.396867 (+ 39us) client_negotiation.cc:769] Sending connection context
0812 02:40:07.396981 (+ 114us) client_negotiation.cc:240] Negotiation successful
0812 02:40:07.397112 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":503,"thread_start_us":557,"threads_started":1}
I20260812 02:40:07.397512 27044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:40:07.381042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53776 (local address 127.7.132.129:41881)
0812 02:40:07.387513 (+ 6471us) server_negotiation.cc:206] Beginning negotiation
0812 02:40:07.387519 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:40:07.387561 (+ 42us) server_negotiation.cc:407] Connection header received
0812 02:40:07.387656 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:40:07.387676 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:40:07.387902 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:40:07.388110 (+ 208us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:40:07.388785 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:40:07.390322 (+ 1537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:40:07.392982 (+ 2660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:40:07.396438 (+ 3456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:40:07.396597 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:40:07.396825 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0812 02:40:07.397025 (+ 200us) server_negotiation.cc:299] Negotiation successful
0812 02:40:07.397187 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6329,"thread_start_us":6159,"threads_started":1}
I20260812 02:40:07.399294 26796 tablet_service.cc:1467] Processing CreateTablet for tablet 65344ea1e1c041f7a399060c05d9835a (DEFAULT_TABLE table=test-table [id=1f474e965151443f961b5e5a52e284ef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:40:07.399294 26949 tablet_service.cc:1467] Processing CreateTablet for tablet 65344ea1e1c041f7a399060c05d9835a (DEFAULT_TABLE table=test-table [id=1f474e965151443f961b5e5a52e284ef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:40:07.400892 26796 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 65344ea1e1c041f7a399060c05d9835a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:40:07.400880 26949 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 65344ea1e1c041f7a399060c05d9835a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:40:07.405344 26641 tablet_service.cc:1467] Processing CreateTablet for tablet 65344ea1e1c041f7a399060c05d9835a (DEFAULT_TABLE table=test-table [id=1f474e965151443f961b5e5a52e284ef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:40:07.406638 26641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 65344ea1e1c041f7a399060c05d9835a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:40:07.416355 27051 tablet_bootstrap.cc:492] T 65344ea1e1c041f7a399060c05d9835a P 6ae48edd11834d13a65cb21c37afb69a: Bootstrap starting.
I20260812 02:40:07.417208 27052 tablet_bootstrap.cc:492] T 65344ea1e1c041f7a399060c05d9835a P c2298009914a4733a6b0604dd0843833: Bootstrap starting.
I20260812 02:40:07.419872 27053 tablet_bootstrap.cc:492] T 65344ea1e1c041f7a399060c05d9835a P cabf179ab4374509b0163a571e20f483: Bootstrap starting.
I20260812 02:40:07.421154 27051 tablet_bootstrap.cc:654] T 65344ea1e1c041f7a399060c05d9835a P 6ae48edd11834d13a65cb21c37afb69a: Neither blocks nor log segments found. Creating new log.
I20260812 02:40:07.421314 27052 tablet_bootstrap.cc:654] T 65344ea1e1c041f7a399060c05d9835a P c2298009914a4733a6b0604dd0843833: Neither blocks nor log segments found. Creating new log.
I20260812 02:40:07.422559 27052 log.cc:824] T 65344ea1e1c041f7a399060c05d9835a P c2298009914a4733a6b0604dd0843833: Log is configured to *not* fsync() on all Append() calls
I20260812 02:40:07.422626 27051 log.cc:824] T 65344ea1e1c041f7a399060c05d9835a P 6ae48edd11834d13a65cb21c37afb69a: Log is configured to *not* fsync() on all Append() calls
I20260812 02:40:07.424000 27053 tablet_bootstrap.cc:654] T 65344ea1e1c041f7a399060c05d9835a P cabf179ab4374509b0163a571e20f483: Neither blocks nor log segments found. Creating new log.
I20260812 02:40:07.425122 27053 log.cc:824] T 65344ea1e1c041f7a399060c05d9835a P cabf179ab4374509b0163a571e20f483: Log is configured to *not* fsync() on all Append() calls
I20260812 02:40:07.425514 27052 tablet_bootstrap.cc:492] T 65344ea1e1c041f7a399060c05d9835a P c2298009914a4733a6b0604dd0843833: No bootstrap required, opened a new log
I20260812 02:40:07.425815 27052 ts_tablet_manager.cc:1397] T 65344ea1e1c041f7a399060c05d9835a P c2298009914a4733a6b0604dd0843833: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260812 02:40:07.426533 27051 tablet_bootstrap.cc:492] T 65344ea1e1c041f7a399060c05d9835a P 6ae48edd11834d13a65cb21c37afb69a: No bootstrap required, opened a new log
I20260812 02:40:07.426802 27051 ts_tablet_manager.cc:1397] T 65344ea1e1c041f7a399060c05d9835a P 6ae48edd11834d13a65cb21c37afb69a: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260812 02:40:07.428233 27053 tablet_bootstrap.cc:492] T 65344ea1e1c041f7a399060c05d9835a P cabf179ab4374509b0163a571e20f483: No bootstrap required, opened a new log
I20260812 02:40:07.428512 27053 ts_tablet_manager.cc:1397] T 65344ea1e1c041f7a399060c05d9835a P cabf179ab4374509b0163a571e20f483: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.006s
I20260812 02:40:07.434204 27052 raft_consensus.cc:348] T 65344ea1e1c041f7a399060c05d9835a P c2298009914a4733a6b0604dd0843833 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cabf179ab4374509b0163a571e20f483" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 41881 } } peers { permanent_uuid: "c2298009914a4733a6b0604dd0843833" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37905 } } peers { permanent_uuid: "6ae48edd11834d13a65cb21c37afb69a" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 34721 } }
I20260812 02:40:07.434593 27052 raft_consensus.cc:374] T 65344ea1e1c041f7a399060c05d9835a P c2298009914a4733a6b0604dd0843833 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:40:07.434715 27052 raft_consensus.cc:729] T 65344ea1e1c041f7a399060c05d9835a P c2298009914a4733a6b0604dd0843833 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2298009914a4733a6b0604dd0843833, State: Initialized, Role: FOLLOWER
I20260812 02:40:07.435325 27052 consensus_queue.cc:260] T 65344ea1e1c041f7a399060c05d9835a P c2298009914a4733a6b0604dd0843833 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cabf179ab4374509b0163a571e20f483" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 41881 } } peers { permanent_uuid: "c2298009914a4733a6b0604dd0843833" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37905 } } peers { permanent_uuid: "6ae48edd11834d13a65cb21c37afb69a" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 34721 } }
I20260812 02:40:07.436439 26871 heartbeater.cc:503] Master 127.7.132.190:42683 was elected leader, sending a full tablet report...
I20260812 02:40:07.437155 27052 ts_tablet_manager.cc:1428] T 65344ea1e1c041f7a399060c05d9835a P c2298009914a4733a6b0604dd0843833: Time spent starting tablet: real 0.011s user 0.010s sys 0.000s
I20260812 02:40:07.437855 27051 raft_consensus.cc:348] T 65344ea1e1c041f7a399060c05d9835a P 6ae48edd11834d13a65cb21c37afb69a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cabf179ab4374509b0163a571e20f483" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 41881 } } peers { permanent_uuid: "c2298009914a4733a6b0604dd0843833" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37905 } } peers { permanent_uuid: "6ae48edd11834d13a65cb21c37afb69a" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 34721 } }
I20260812 02:40:07.438421 27051 raft_consensus.cc:374] T 65344ea1e1c041f7a399060c05d9835a P 6ae48edd11834d13a65cb21c37afb69a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:40:07.438530 27051 raft_consensus.cc:729] T 65344ea1e1c041f7a399060c05d9835a P 6ae48edd11834d13a65cb21c37afb69a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ae48edd11834d13a65cb21c37afb69a, State: Initialized, Role: FOLLOWER
I20260812 02:40:07.439172 27051 consensus_queue.cc:260] T 65344ea1e1c041f7a399060c05d9835a P 6ae48edd11834d13a65cb21c37afb69a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cabf179ab4374509b0163a571e20f483" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 41881 } } peers { permanent_uuid: "c2298009914a4733a6b0604dd0843833" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37905 } } peers { permanent_uuid: "6ae48edd11834d13a65cb21c37afb69a" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 34721 } }
I20260812 02:40:07.439205 27053 raft_consensus.cc:348] T 65344ea1e1c041f7a399060c05d9835a P cabf179ab4374509b0163a571e20f483 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cabf179ab4374509b0163a571e20f483" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 41881 } } peers { permanent_uuid: "c2298009914a4733a6b0604dd0843833" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37905 } } peers { permanent_uuid: "6ae48edd11834d13a65cb21c37afb69a" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 34721 } }
I20260812 02:40:07.439630 27053 raft_consensus.cc:374] T 65344ea1e1c041f7a399060c05d9835a P cabf179ab4374509b0163a571e20f483 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:40:07.439841 27053 raft_consensus.cc:729] T 65344ea1e1c041f7a399060c05d9835a P cabf179ab4374509b0163a571e20f483 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cabf179ab4374509b0163a571e20f483, State: Initialized, Role: FOLLOWER
I20260812 02:40:07.440587 27053 consensus_queue.cc:260] T 65344ea1e1c041f7a399060c05d9835a P cabf179ab4374509b0163a571e20f483 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cabf179ab4374509b0163a571e20f483" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 41881 } } peers { permanent_uuid: "c2298009914a4733a6b0604dd0843833" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37905 } } peers { permanent_uuid: "6ae48edd11834d13a65cb21c37afb69a" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 34721 } }
I20260812 02:40:07.440773 27027 heartbeater.cc:503] Master 127.7.132.190:42683 was elected leader, sending a full tablet report...
I20260812 02:40:07.441206 27051 ts_tablet_manager.cc:1428] T 65344ea1e1c041f7a399060c05d9835a P 6ae48edd11834d13a65cb21c37afb69a: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
I20260812 02:40:07.442510 27053 ts_tablet_manager.cc:1428] T 65344ea1e1c041f7a399060c05d9835a P cabf179ab4374509b0163a571e20f483: Time spent starting tablet: real 0.014s user 0.011s sys 0.004s
W20260812 02:40:07.473640 26715 tablet.cc:2369] T 65344ea1e1c041f7a399060c05d9835a P cabf179ab4374509b0163a571e20f483: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 02:40:07.525177 26872 tablet.cc:2369] T 65344ea1e1c041f7a399060c05d9835a P c2298009914a4733a6b0604dd0843833: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 02:40:07.553853 27059 raft_consensus.cc:482] T 65344ea1e1c041f7a399060c05d9835a P c2298009914a4733a6b0604dd0843833 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 02:40:07.554232 27059 raft_consensus.cc:504] T 65344ea1e1c041f7a399060c05d9835a P c2298009914a4733a6b0604dd0843833 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cabf179ab4374509b0163a571e20f483" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 41881 } } peers { permanent_uuid: "c2298009914a4733a6b0604dd0843833" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37905 } } peers { permanent_uuid: "6ae48edd11834d13a65cb21c37afb69a" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 34721 } }
I20260812 02:40:07.556177 27059 leader_election.cc:288] T 65344ea1e1c041f7a399060c05d9835a P c2298009914a4733a6b0604dd0843833 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cabf179ab4374509b0163a571e20f483 (127.7.132.129:41881), 6ae48edd11834d13a65cb21c37afb69a (127.7.132.131:34721)
W20260812 02:40:07.558594 27028 tablet.cc:2369] T 65344ea1e1c041f7a399060c05d9835a P 6ae48edd11834d13a65cb21c37afb69a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 02:40:07.563905 27044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:40:07.556662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:47553 (local address 127.7.132.129:41881)
0812 02:40:07.556904 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0812 02:40:07.556926 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0812 02:40:07.557108 (+ 182us) server_negotiation.cc:407] Connection header received
0812 02:40:07.557361 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:40:07.557370 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:40:07.557477 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:40:07.557577 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:40:07.558313 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:40:07.559309 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:40:07.561797 (+ 2488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:40:07.563059 (+ 1262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:40:07.563233 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:40:07.563319 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0812 02:40:07.563565 (+ 246us) server_negotiation.cc:299] Negotiation successful
0812 02:40:07.563706 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260812 02:40:07.564141 27063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:40:07.556395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:41881 (local address 127.7.132.130:47553)
0812 02:40:07.556984 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0812 02:40:07.557000 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 02:40:07.557179 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:40:07.557627 (+ 448us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:40:07.557633 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:40:07.557659 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:40:07.558151 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:40:07.558163 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:40:07.559454 (+ 1291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:40:07.559461 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:40:07.561622 (+ 2161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:40:07.561638 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:40:07.563256 (+ 1618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:40:07.563261 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:40:07.563382 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:40:07.563405 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 02:40:07.563513 (+ 108us) client_negotiation.cc:240] Negotiation successful
0812 02:40:07.563674 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":480,"mutex_wait_us":1,"thread_start_us":319,"threads_started":1}
I20260812 02:40:07.564488 27050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:40:07.557296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:36717 (local address 127.7.132.131:34721)
0812 02:40:07.557518 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0812 02:40:07.557526 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:40:07.557899 (+ 373us) server_negotiation.cc:407] Connection header received
0812 02:40:07.558055 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:40:07.558059 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:40:07.558162 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:40:07.558259 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:40:07.558974 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:40:07.559866 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:40:07.562183 (+ 2317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:40:07.563400 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:40:07.563560 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:40:07.563628 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 02:40:07.563962 (+ 334us) server_negotiation.cc:299] Negotiation successful
0812 02:40:07.564225 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260812 02:40:07.564517 27064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:40:07.557170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.131:34721 (local address 127.7.132.130:36717)
0812 02:40:07.557788 (+ 618us) negotiation.cc:107] Waiting for socket to connect
0812 02:40:07.557809 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 02:40:07.557941 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:40:07.558287 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:40:07.558294 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:40:07.558323 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:40:07.558735 (+ 412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:40:07.558748 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:40:07.559994 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:40:07.560004 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:40:07.561986 (+ 1982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:40:07.561996 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:40:07.563599 (+ 1603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:40:07.563605 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:40:07.563777 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:40:07.563825 (+ 48us) client_negotiation.cc:769] Sending connection context
0812 02:40:07.563915 (+ 90us) client_negotiation.cc:240] Negotiation successful
0812 02:40:07.564031 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":479,"mutex_wait_us":46,"thread_start_us":276,"threads_started":1}
I20260812 02:40:07.565320 26663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65344ea1e1c041f7a399060c05d9835a" candidate_uuid: "c2298009914a4733a6b0604dd0843833" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cabf179ab4374509b0163a571e20f483" is_pre_election: true
I20260812 02:40:07.565558 26973 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65344ea1e1c041f7a399060c05d9835a" candidate_uuid: "c2298009914a4733a6b0604dd0843833" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ae48edd11834d13a65cb21c37afb69a" is_pre_election: true
I20260812 02:40:07.565913 26663 raft_consensus.cc:2454] T 65344ea1e1c041f7a399060c05d9835a P cabf179ab4374509b0163a571e20f483 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2298009914a4733a6b0604dd0843833 in term 0.
I20260812 02:40:07.566154 26973 raft_consensus.cc:2454] T 65344ea1e1c041f7a399060c05d9835a P 6ae48edd11834d13a65cb21c37afb69a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2298009914a4733a6b0604dd0843833 in term 0.
I20260812 02:40:07.566730 26743 leader_election.cc:302] T 65344ea1e1c041f7a399060c05d9835a P c2298009914a4733a6b0604dd0843833 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c2298009914a4733a6b0604dd0843833, cabf179ab4374509b0163a571e20f483; no voters:
I20260812 02:40:07.567070 27059 raft_consensus.cc:2781] T 65344ea1e1c041f7a399060c05d9835a P c2298009914a4733a6b0604dd0843833 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 02:40:07.567209 27059 raft_consensus.cc:482] T 65344ea1e1c041f7a399060c05d9835a P c2298009914a4733a6b0604dd0843833 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 02:40:07.567330 27059 raft_consensus.cc:3037] T 65344ea1e1c041f7a399060c05d9835a P c2298009914a4733a6b0604dd0843833 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:40:07.569528 27059 raft_consensus.cc:504] T 65344ea1e1c041f7a399060c05d9835a P c2298009914a4733a6b0604dd0843833 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cabf179ab4374509b0163a571e20f483" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 41881 } } peers { permanent_uuid: "c2298009914a4733a6b0604dd0843833" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37905 } } peers { permanent_uuid: "6ae48edd11834d13a65cb21c37afb69a" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 34721 } }
I20260812 02:40:07.570155 27059 leader_election.cc:288] T 65344ea1e1c041f7a399060c05d9835a P c2298009914a4733a6b0604dd0843833 [CANDIDATE]: Term 1 election: Requested vote from peers cabf179ab4374509b0163a571e20f483 (127.7.132.129:41881), 6ae48edd11834d13a65cb21c37afb69a (127.7.132.131:34721)
I20260812 02:40:07.570551 26663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65344ea1e1c041f7a399060c05d9835a" candidate_uuid: "c2298009914a4733a6b0604dd0843833" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cabf179ab4374509b0163a571e20f483"
I20260812 02:40:07.570654 26973 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65344ea1e1c041f7a399060c05d9835a" candidate_uuid: "c2298009914a4733a6b0604dd0843833" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ae48edd11834d13a65cb21c37afb69a"
I20260812 02:40:07.570740 26663 raft_consensus.cc:3037] T 65344ea1e1c041f7a399060c05d9835a P cabf179ab4374509b0163a571e20f483 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:40:07.570878 26973 raft_consensus.cc:3037] T 65344ea1e1c041f7a399060c05d9835a P 6ae48edd11834d13a65cb21c37afb69a [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:40:07.573339 26663 raft_consensus.cc:2454] T 65344ea1e1c041f7a399060c05d9835a P cabf179ab4374509b0163a571e20f483 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2298009914a4733a6b0604dd0843833 in term 1.
I20260812 02:40:07.573529 26973 raft_consensus.cc:2454] T 65344ea1e1c041f7a399060c05d9835a P 6ae48edd11834d13a65cb21c37afb69a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2298009914a4733a6b0604dd0843833 in term 1.
I20260812 02:40:07.573742 26743 leader_election.cc:302] T 65344ea1e1c041f7a399060c05d9835a P c2298009914a4733a6b0604dd0843833 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c2298009914a4733a6b0604dd0843833, cabf179ab4374509b0163a571e20f483; no voters:
I20260812 02:40:07.574028 27059 raft_consensus.cc:2781] T 65344ea1e1c041f7a399060c05d9835a P c2298009914a4733a6b0604dd0843833 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:40:07.574502 27059 raft_consensus.cc:686] T 65344ea1e1c041f7a399060c05d9835a P c2298009914a4733a6b0604dd0843833 [term 1 LEADER]: Becoming Leader. State: Replica: c2298009914a4733a6b0604dd0843833, State: Running, Role: LEADER
I20260812 02:40:07.575057 27059 consensus_queue.cc:237] T 65344ea1e1c041f7a399060c05d9835a P c2298009914a4733a6b0604dd0843833 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cabf179ab4374509b0163a571e20f483" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 41881 } } peers { permanent_uuid: "c2298009914a4733a6b0604dd0843833" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37905 } } peers { permanent_uuid: "6ae48edd11834d13a65cb21c37afb69a" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 34721 } }
I20260812 02:40:07.579550 26487 catalog_manager.cc:5495] T 65344ea1e1c041f7a399060c05d9835a P c2298009914a4733a6b0604dd0843833 reported cstate change: term changed from 0 to 1, leader changed from <none> to c2298009914a4733a6b0604dd0843833 (127.7.132.130). New cstate: current_term: 1 leader_uuid: "c2298009914a4733a6b0604dd0843833" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cabf179ab4374509b0163a571e20f483" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 41881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2298009914a4733a6b0604dd0843833" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37905 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ae48edd11834d13a65cb21c37afb69a" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 34721 } health_report { overall_health: UNKNOWN } } }
I20260812 02:40:07.631023 27048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:40:07.625532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55180 (local address 127.7.132.130:37905)
0812 02:40:07.625780 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0812 02:40:07.625786 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:40:07.625808 (+ 22us) server_negotiation.cc:407] Connection header received
0812 02:40:07.625926 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:40:07.625931 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:40:07.626029 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:40:07.626210 (+ 181us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 02:40:07.626601 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:40:07.627321 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:40:07.628874 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:40:07.629271 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:40:07.629401 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:40:07.629869 (+ 468us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 02:40:07.630441 (+ 572us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 02:40:07.630596 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0812 02:40:07.630658 (+ 62us) server_negotiation.cc:299] Negotiation successful
0812 02:40:07.630737 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260812 02:40:07.639946 26973 raft_consensus.cc:1261] T 65344ea1e1c041f7a399060c05d9835a P 6ae48edd11834d13a65cb21c37afb69a [term 1 FOLLOWER]: Refusing update from remote peer c2298009914a4733a6b0604dd0843833: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:40:07.641582 26663 raft_consensus.cc:1261] T 65344ea1e1c041f7a399060c05d9835a P cabf179ab4374509b0163a571e20f483 [term 1 FOLLOWER]: Refusing update from remote peer c2298009914a4733a6b0604dd0843833: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:40:07.642808 27065 consensus_queue.cc:1035] T 65344ea1e1c041f7a399060c05d9835a P c2298009914a4733a6b0604dd0843833 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cabf179ab4374509b0163a571e20f483" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 41881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:40:07.643739 27065 consensus_queue.cc:1035] T 65344ea1e1c041f7a399060c05d9835a P c2298009914a4733a6b0604dd0843833 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ae48edd11834d13a65cb21c37afb69a" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 34721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:40:07.659276 27068 mvcc.cc:205] Tried to move back new op lower bound from 7317513861680828416 to 7317513861431152640. Current Snapshot: MvccSnapshot[applied={T|T < 7317513861680828416}]
I20260812 02:40:07.660439 27070 mvcc.cc:205] Tried to move back new op lower bound from 7317513861680828416 to 7317513861431152640. Current Snapshot: MvccSnapshot[applied={T|T < 7317513861680828416}]
I20260812 02:40:07.663220 27071 mvcc.cc:205] Tried to move back new op lower bound from 7317513861680828416 to 7317513861431152640. Current Snapshot: MvccSnapshot[applied={T|T < 7317513861680828416}]
I20260812 02:40:07.683892 26487 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42014:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 02:40:07.684228 26487 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42014:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 02:40:07.688827 26487 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0fad46d3a46c4597903b423dfd124d70: Sending DeleteTablet for 3 replicas of tablet 65344ea1e1c041f7a399060c05d9835a
I20260812 02:40:07.690109 26641 tablet_service.cc:1514] Processing DeleteTablet for tablet 65344ea1e1c041f7a399060c05d9835a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:40:07 UTC) from {username='kudu', principal='kudu/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:53776
I20260812 02:40:07.690404 26949 tablet_service.cc:1514] Processing DeleteTablet for tablet 65344ea1e1c041f7a399060c05d9835a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:40:07 UTC) from {username='kudu', principal='kudu/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:38538
I20260812 02:40:07.691484 26796 tablet_service.cc:1514] Processing DeleteTablet for tablet 65344ea1e1c041f7a399060c05d9835a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:40:07 UTC) from {username='kudu', principal='kudu/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:55166
I20260812 02:40:07.698158 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 26558
I20260812 02:40:07.701296 27080 tablet_replica.cc:330] T 65344ea1e1c041f7a399060c05d9835a P cabf179ab4374509b0163a571e20f483: stopping tablet replica
I20260812 02:40:07.701992 27080 raft_consensus.cc:2229] T 65344ea1e1c041f7a399060c05d9835a P cabf179ab4374509b0163a571e20f483 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 02:40:07.702751 27080 raft_consensus.cc:2258] T 65344ea1e1c041f7a399060c05d9835a P cabf179ab4374509b0163a571e20f483 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:40:07.705312 27080 ts_tablet_manager.cc:1905] T 65344ea1e1c041f7a399060c05d9835a P cabf179ab4374509b0163a571e20f483: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:40:07.714017 27080 ts_tablet_manager.cc:1918] T 65344ea1e1c041f7a399060c05d9835a P cabf179ab4374509b0163a571e20f483: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 02:40:07.714389 27080 log.cc:1196] T 65344ea1e1c041f7a399060c05d9835a P cabf179ab4374509b0163a571e20f483: Deleting WAL directory at /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-0/wal/wals/65344ea1e1c041f7a399060c05d9835a
I20260812 02:40:07.715098 27080 ts_tablet_manager.cc:1939] T 65344ea1e1c041f7a399060c05d9835a P cabf179ab4374509b0163a571e20f483: Deleting consensus metadata
I20260812 02:40:07.717342 26469 catalog_manager.cc:4841] TS cabf179ab4374509b0163a571e20f483 (127.7.132.129:41881): tablet 65344ea1e1c041f7a399060c05d9835a (table test-table [id=1f474e965151443f961b5e5a52e284ef]) successfully deleted
I20260812 02:40:07.718925 27082 tablet_replica.cc:330] T 65344ea1e1c041f7a399060c05d9835a P c2298009914a4733a6b0604dd0843833: stopping tablet replica
I20260812 02:40:07.719553 27082 raft_consensus.cc:2229] T 65344ea1e1c041f7a399060c05d9835a P c2298009914a4733a6b0604dd0843833 [term 1 LEADER]: Raft consensus shutting down.
I20260812 02:40:07.720242 27081 tablet_replica.cc:330] T 65344ea1e1c041f7a399060c05d9835a P 6ae48edd11834d13a65cb21c37afb69a: stopping tablet replica
I20260812 02:40:07.720999 27082 raft_consensus.cc:2258] T 65344ea1e1c041f7a399060c05d9835a P c2298009914a4733a6b0604dd0843833 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:40:07.721617 27081 raft_consensus.cc:2229] T 65344ea1e1c041f7a399060c05d9835a P 6ae48edd11834d13a65cb21c37afb69a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 02:40:07.722582 27081 raft_consensus.cc:2258] T 65344ea1e1c041f7a399060c05d9835a P 6ae48edd11834d13a65cb21c37afb69a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:40:07.725066 27082 ts_tablet_manager.cc:1905] T 65344ea1e1c041f7a399060c05d9835a P c2298009914a4733a6b0604dd0843833: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:40:07.727140 27081 ts_tablet_manager.cc:1905] T 65344ea1e1c041f7a399060c05d9835a P 6ae48edd11834d13a65cb21c37afb69a: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 12 02:40:07 dist-test-slave-mn4r krb5kdc[26436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502405, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.132.129@KRBTEST.COM
I20260812 02:40:07.735271 27082 ts_tablet_manager.cc:1918] T 65344ea1e1c041f7a399060c05d9835a P c2298009914a4733a6b0604dd0843833: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 02:40:07.735558 27082 log.cc:1196] T 65344ea1e1c041f7a399060c05d9835a P c2298009914a4733a6b0604dd0843833: Deleting WAL directory at /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-1/wal/wals/65344ea1e1c041f7a399060c05d9835a
I20260812 02:40:07.736289 27082 ts_tablet_manager.cc:1939] T 65344ea1e1c041f7a399060c05d9835a P c2298009914a4733a6b0604dd0843833: Deleting consensus metadata
I20260812 02:40:07.738415 26473 catalog_manager.cc:4841] TS c2298009914a4733a6b0604dd0843833 (127.7.132.130:37905): tablet 65344ea1e1c041f7a399060c05d9835a (table test-table [id=1f474e965151443f961b5e5a52e284ef]) successfully deleted
I20260812 02:40:07.743397 27081 ts_tablet_manager.cc:1918] T 65344ea1e1c041f7a399060c05d9835a P 6ae48edd11834d13a65cb21c37afb69a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 02:40:07.743707 27081 log.cc:1196] T 65344ea1e1c041f7a399060c05d9835a P 6ae48edd11834d13a65cb21c37afb69a: Deleting WAL directory at /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-2/wal/wals/65344ea1e1c041f7a399060c05d9835a
I20260812 02:40:07.744078 27044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:40:07.699371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53782 (local address 127.7.132.129:41881)
0812 02:40:07.699556 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0812 02:40:07.699563 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:40:07.718106 (+ 18543us) server_negotiation.cc:407] Connection header received
0812 02:40:07.718344 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:40:07.718351 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:40:07.718437 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:40:07.718554 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:40:07.720795 (+ 2241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:40:07.721980 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:40:07.724166 (+ 2186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:40:07.724805 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:40:07.724901 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:40:07.735904 (+ 11003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:40:07.735935 (+ 31us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:40:07.735954 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:40:07.736001 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:40:07.740057 (+ 4056us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:40:07.741140 (+ 1083us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:40:07.741149 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:40:07.741152 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:40:07.741318 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:40:07.741845 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:40:07.741852 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:40:07.741857 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:40:07.742454 (+ 597us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:40:07.742768 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0812 02:40:07.743438 (+ 670us) server_negotiation.cc:299] Negotiation successful
0812 02:40:07.743673 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260812 02:40:07.744710 27081 ts_tablet_manager.cc:1939] T 65344ea1e1c041f7a399060c05d9835a P 6ae48edd11834d13a65cb21c37afb69a: Deleting consensus metadata
I20260812 02:40:07.745577 26708 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:40:07.747043 26469 catalog_manager.cc:4841] TS 6ae48edd11834d13a65cb21c37afb69a (127.7.132.131:34721): tablet 65344ea1e1c041f7a399060c05d9835a (table test-table [id=1f474e965151443f961b5e5a52e284ef]) successfully deleted
I20260812 02:40:07.897783 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 26558
I20260812 02:40:07.934739 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 26721
Aug 12 02:40:07 dist-test-slave-mn4r krb5kdc[26436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502405, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.132.130@KRBTEST.COM
I20260812 02:40:07.961650 27048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:40:07.935748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55184 (local address 127.7.132.130:37905)
0812 02:40:07.936403 (+ 655us) server_negotiation.cc:206] Beginning negotiation
0812 02:40:07.936409 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:40:07.936427 (+ 18us) server_negotiation.cc:407] Connection header received
0812 02:40:07.936493 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:40:07.936498 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:40:07.936578 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:40:07.936692 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:40:07.939226 (+ 2534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:40:07.940455 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:40:07.943381 (+ 2926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:40:07.944091 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:40:07.944223 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:40:07.951586 (+ 7363us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:40:07.951616 (+ 30us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:40:07.951692 (+ 76us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:40:07.951757 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:40:07.957489 (+ 5732us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:40:07.958501 (+ 1012us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:40:07.958516 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:40:07.958522 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:40:07.958775 (+ 253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:40:07.959276 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:40:07.959284 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:40:07.959290 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:40:07.959964 (+ 674us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:40:07.960417 (+ 453us) server_negotiation.cc:1035] Waiting for connection context
0812 02:40:07.960695 (+ 278us) server_negotiation.cc:299] Negotiation successful
0812 02:40:07.961037 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":512}
I20260812 02:40:07.963194 26864 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:40:08.077433 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 26721
I20260812 02:40:08.101483 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 26877
Aug 12 02:40:08 dist-test-slave-mn4r krb5kdc[26436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502405, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.132.131@KRBTEST.COM
I20260812 02:40:08.116840 27089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:40:08.102288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38554 (local address 127.7.132.131:34721)
0812 02:40:08.102776 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0812 02:40:08.102781 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:40:08.102798 (+ 17us) server_negotiation.cc:407] Connection header received
0812 02:40:08.102856 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:40:08.102859 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:40:08.102917 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:40:08.103010 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:40:08.104320 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:40:08.105060 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:40:08.106434 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:40:08.106811 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:40:08.106906 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:40:08.111653 (+ 4747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:40:08.111667 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:40:08.111683 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:40:08.111723 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:40:08.114414 (+ 2691us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:40:08.115095 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:40:08.115102 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:40:08.115104 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:40:08.115231 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:40:08.115571 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:40:08.115574 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:40:08.115576 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:40:08.115910 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:40:08.116148 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0812 02:40:08.116394 (+ 246us) server_negotiation.cc:299] Negotiation successful
0812 02:40:08.116565 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":390,"thread_start_us":280,"threads_started":1}
I20260812 02:40:08.117600 27021 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:40:08.209661 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 26877
I20260812 02:40:08.230875 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 26450
I20260812 02:40:08.232123 26522 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:40:08.346324 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 26450
I20260812 02:40:28.374948 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:42683
--webserver_interface=127.7.132.190
--webserver_port=41403
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:44005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.132.190:42683
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:40:28.741340 27092 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:40:28.741731 27092 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:40:28.741927 27092 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:40:28.749301 27092 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260812 02:40:28.749431 27092 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260812 02:40:28.749503 27092 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260812 02:40:28.749606 27092 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:40:28.749712 27092 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:40:28.749828 27092 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:40:28.749900 27092 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:40:28.749980 27092 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:40:28.760035 27092 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:44005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.7.132.190:42683
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:42683
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.132.190
--webserver_port=41403
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:40:28.761375 27092 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:40:28.762943 27092 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:40:28.790393 27100 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:40:28.790486 27102 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:40:28.790393 27107 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:40:28.790807 27092 server_base.cc:1034] running on GCE node
I20260812 02:40:28.792129 27092 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:40:28.793902 27092 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:40:28.795167 27092 hybrid_clock.cc:648] HybridClock initialized: now 1786502428795096 us; error 69 us; skew 500 ppm
I20260812 02:40:28.795605 27092 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:40:28 dist-test-slave-mn4r krb5kdc[26436](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502428, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:40:28.804482 27092 init.cc:377] Logged in from keytab as kudu/127.7.132.190@KRBTEST.COM (short username kudu)
I20260812 02:40:28.806444 27092 webserver.cc:460] Webserver started at http://127.7.132.190:41403/ using document root <none> and password file <none>
I20260812 02:40:28.807284 27092 fs_manager.cc:359] Metadata directory not provided
I20260812 02:40:28.807446 27092 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:40:28.813908 27092 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.003s
I20260812 02:40:28.817124 27114 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:40:28.818601 27092 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 02:40:28.818804 27092 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/master-0/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/master-0/wal
uuid: "0fad46d3a46c4597903b423dfd124d70"
format_stamp: "Formatted at 2026-08-12 02:40:05 on dist-test-slave-mn4r"
server_key: "b731db17ef27b79895faecff3c1d5c27"
server_key_iv: "ed35bab47187d559b4b58d26a970b636"
server_key_version: "encryptionkey@0"
I20260812 02:40:28.819918 27092 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:40:28.848887 27092 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:40:28.853343 27092 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:40:28.853673 27092 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:40:28.874101 27092 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.190:42683
I20260812 02:40:28.874192 27174 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.190:42683 every 8 connection(s)
I20260812 02:40:28.875856 27092 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
I20260812 02:40:28.878239 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 27092
I20260812 02:40:28.879127 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.129:41881
--local_ip_for_outbound_sockets=127.7.132.129
--tserver_master_addrs=127.7.132.190:42683
--webserver_port=33519
--webserver_interface=127.7.132.129
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.129.keytab
--principal=kudu/127.7.132.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:44005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 02:40:28.883635 27175 sys_catalog.cc:263] Verifying existing consensus state
I20260812 02:40:28.887154 27175 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0fad46d3a46c4597903b423dfd124d70: Bootstrap starting.
I20260812 02:40:28.908190 27175 log.cc:824] T 00000000000000000000000000000000 P 0fad46d3a46c4597903b423dfd124d70: Log is configured to *not* fsync() on all Append() calls
I20260812 02:40:28.921200 27175 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0fad46d3a46c4597903b423dfd124d70: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 02:40:28.921785 27175 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0fad46d3a46c4597903b423dfd124d70: Bootstrap complete.
I20260812 02:40:28.931808 27175 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0fad46d3a46c4597903b423dfd124d70 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fad46d3a46c4597903b423dfd124d70" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 42683 } }
I20260812 02:40:28.932703 27175 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0fad46d3a46c4597903b423dfd124d70 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fad46d3a46c4597903b423dfd124d70, State: Initialized, Role: FOLLOWER
I20260812 02:40:28.933342 27175 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0fad46d3a46c4597903b423dfd124d70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fad46d3a46c4597903b423dfd124d70" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 42683 } }
I20260812 02:40:28.933606 27175 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0fad46d3a46c4597903b423dfd124d70 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 02:40:28.933746 27175 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0fad46d3a46c4597903b423dfd124d70 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 02:40:28.933923 27175 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0fad46d3a46c4597903b423dfd124d70 [term 1 FOLLOWER]: Advancing to term 2
I20260812 02:40:28.937916 27175 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0fad46d3a46c4597903b423dfd124d70 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fad46d3a46c4597903b423dfd124d70" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 42683 } }
I20260812 02:40:28.938515 27175 leader_election.cc:302] T 00000000000000000000000000000000 P 0fad46d3a46c4597903b423dfd124d70 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0fad46d3a46c4597903b423dfd124d70; no voters:
I20260812 02:40:28.939250 27175 leader_election.cc:288] T 00000000000000000000000000000000 P 0fad46d3a46c4597903b423dfd124d70 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 02:40:28.939404 27181 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0fad46d3a46c4597903b423dfd124d70 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 02:40:28.940636 27181 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0fad46d3a46c4597903b423dfd124d70 [term 2 LEADER]: Becoming Leader. State: Replica: 0fad46d3a46c4597903b423dfd124d70, State: Running, Role: LEADER
I20260812 02:40:28.941377 27181 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0fad46d3a46c4597903b423dfd124d70 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fad46d3a46c4597903b423dfd124d70" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 42683 } }
I20260812 02:40:28.941973 27175 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0fad46d3a46c4597903b423dfd124d70 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:40:28.944193 27182 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0fad46d3a46c4597903b423dfd124d70 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "0fad46d3a46c4597903b423dfd124d70" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fad46d3a46c4597903b423dfd124d70" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 42683 } } }
I20260812 02:40:28.944651 27182 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0fad46d3a46c4597903b423dfd124d70 [sys.catalog]: This master's current role is: LEADER
I20260812 02:40:28.948518 27183 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0fad46d3a46c4597903b423dfd124d70 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0fad46d3a46c4597903b423dfd124d70. Latest consensus state: current_term: 2 leader_uuid: "0fad46d3a46c4597903b423dfd124d70" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fad46d3a46c4597903b423dfd124d70" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 42683 } } }
I20260812 02:40:28.949033 27183 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0fad46d3a46c4597903b423dfd124d70 [sys.catalog]: This master's current role is: LEADER
I20260812 02:40:28.950776 27187 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:40:28.959143 27187 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 02:40:28.960590 27187 catalog_manager.cc:1259] Loaded cluster ID: 40c1dacf063b4f2d90266628e0db9540
I20260812 02:40:28.960765 27187 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 02:40:28.964990 27187 catalog_manager.cc:1504] Loading token signing keys...
I20260812 02:40:28.966751 27187 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 0fad46d3a46c4597903b423dfd124d70: Loaded TSK: 0
I20260812 02:40:28.988452 27199 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260812 02:40:29.013355 27187 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0fad46d3a46c4597903b423dfd124d70: Generated new TSK 1
I20260812 02:40:29.013782 27187 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 02:40:29.273305 27179 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:40:29.273602 27179 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:40:29.273753 27179 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:40:29.281289 27179 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:40:29.281414 27179 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:40:29.281592 27179 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.129
I20260812 02:40:29.292516 27179 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:44005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.129.keytab
--principal=kudu/127.7.132.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.129:41881
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.132.129
--webserver_port=33519
--tserver_master_addrs=127.7.132.190:42683
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.129
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:40:29.294026 27179 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:40:29.295480 27179 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:40:29.312026 27208 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:40:29.311985 27209 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:40:29.312776 27211 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:40:29.313278 27179 server_base.cc:1034] running on GCE node
I20260812 02:40:29.313980 27179 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:40:29.315099 27179 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:40:29.316344 27179 hybrid_clock.cc:648] HybridClock initialized: now 1786502429316267 us; error 72 us; skew 500 ppm
I20260812 02:40:29.316704 27179 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:40:29 dist-test-slave-mn4r krb5kdc[26436](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502429, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:40:29.320938 27179 init.cc:377] Logged in from keytab as kudu/127.7.132.129@KRBTEST.COM (short username kudu)
I20260812 02:40:29.322890 27179 webserver.cc:460] Webserver started at http://127.7.132.129:33519/ using document root <none> and password file <none>
I20260812 02:40:29.323720 27179 fs_manager.cc:359] Metadata directory not provided
I20260812 02:40:29.323897 27179 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:40:29.329977 27179 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260812 02:40:29.333072 27218 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:40:29.334610 27179 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 02:40:29.334797 27179 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-0/wal
uuid: "cabf179ab4374509b0163a571e20f483"
format_stamp: "Formatted at 2026-08-12 02:40:06 on dist-test-slave-mn4r"
server_key: "dfe9a185d5e7020d0b76ae28c460f70b"
server_key_iv: "77e6c572a50b660125dad0e8fca42854"
server_key_version: "encryptionkey@0"
I20260812 02:40:29.335820 27179 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:40:29.353792 27179 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:40:29.359169 27179 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:40:29.359479 27179 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:40:29.360641 27179 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:40:29.362342 27179 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:40:29.362468 27179 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:40:29.362605 27179 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:40:29.362694 27179 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:40:29.398660 27179 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.129:41881
I20260812 02:40:29.398751 27349 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.129:41881 every 8 connection(s)
I20260812 02:40:29.400180 27179 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-0/data/info.pb
I20260812 02:40:29.402405 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 27179
I20260812 02:40:29.403250 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.130:37905
--local_ip_for_outbound_sockets=127.7.132.130
--tserver_master_addrs=127.7.132.190:42683
--webserver_port=36611
--webserver_interface=127.7.132.130
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.130.keytab
--principal=kudu/127.7.132.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:44005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 02:40:29 dist-test-slave-mn4r krb5kdc[26436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502429, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.129@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 02:40:29.425016 27353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:40:29.403447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:48289 (local address 127.7.132.190:42683)
0812 02:40:29.404197 (+ 750us) server_negotiation.cc:206] Beginning negotiation
0812 02:40:29.404217 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0812 02:40:29.404722 (+ 505us) server_negotiation.cc:407] Connection header received
0812 02:40:29.406933 (+ 2211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:40:29.406945 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:40:29.407318 (+ 373us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:40:29.407681 (+ 363us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:40:29.409227 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:40:29.410710 (+ 1483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:40:29.412247 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:40:29.412872 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:40:29.413076 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:40:29.417129 (+ 4053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:40:29.417137 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:40:29.417151 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:40:29.417187 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:40:29.420250 (+ 3063us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:40:29.420990 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:40:29.420997 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:40:29.421007 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:40:29.421177 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:40:29.421620 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:40:29.421627 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:40:29.421633 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:40:29.422213 (+ 580us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:40:29.422425 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0812 02:40:29.423225 (+ 800us) server_negotiation.cc:299] Negotiation successful
0812 02:40:29.423628 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":492,"thread_start_us":343,"threads_started":1}
I20260812 02:40:29.425009 27354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:40:29.403673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:42683 (local address 127.7.132.129:48289)
0812 02:40:29.404534 (+ 861us) negotiation.cc:107] Waiting for socket to connect
0812 02:40:29.404572 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 02:40:29.406386 (+ 1814us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:40:29.407834 (+ 1448us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:40:29.407844 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:40:29.408457 (+ 613us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:40:29.408988 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:40:29.409005 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:40:29.410879 (+ 1874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:40:29.410885 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:40:29.412099 (+ 1214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:40:29.412113 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:40:29.413112 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:40:29.413120 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:40:29.413439 (+ 319us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:40:29.414292 (+ 853us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:40:29.414326 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:40:29.416824 (+ 2498us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:40:29.420395 (+ 3571us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:40:29.420404 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:40:29.420418 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:40:29.420845 (+ 427us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:40:29.421299 (+ 454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:40:29.421304 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:40:29.421308 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:40:29.421502 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:40:29.422370 (+ 868us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:40:29.422377 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:40:29.422719 (+ 342us) client_negotiation.cc:769] Sending connection context
0812 02:40:29.422995 (+ 276us) client_negotiation.cc:240] Negotiation successful
0812 02:40:29.423628 (+ 633us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":618,"thread_start_us":404,"threads_started":1}
I20260812 02:40:29.428558 27350 heartbeater.cc:347] Connected to a master server at 127.7.132.190:42683
I20260812 02:40:29.428891 27350 heartbeater.cc:464] Registering TS with master...
I20260812 02:40:29.429630 27350 heartbeater.cc:511] Master 127.7.132.190:42683 requested a full tablet report, sending...
I20260812 02:40:29.432348 27132 ts_manager.cc:194] Registered new tserver with Master: cabf179ab4374509b0163a571e20f483 (127.7.132.129:41881)
I20260812 02:40:29.434871 27132 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.129@KRBTEST.COM'} at 127.7.132.129:48289
W20260812 02:40:29.757872 27355 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:40:29.758244 27355 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:40:29.758360 27355 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:40:29.766187 27355 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:40:29.766297 27355 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:40:29.766412 27355 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.130
I20260812 02:40:29.776145 27355 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:44005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.130.keytab
--principal=kudu/127.7.132.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.130:37905
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.132.130
--webserver_port=36611
--tserver_master_addrs=127.7.132.190:42683
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.130
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:40:29.777427 27355 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:40:29.778844 27355 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:40:29.792778 27361 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:40:29.792810 27363 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:40:29.792709 27365 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:40:29.794023 27355 server_base.cc:1034] running on GCE node
I20260812 02:40:29.794808 27355 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:40:29.795872 27355 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:40:29.797092 27355 hybrid_clock.cc:648] HybridClock initialized: now 1786502429797022 us; error 54 us; skew 500 ppm
I20260812 02:40:29.797449 27355 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:40:29 dist-test-slave-mn4r krb5kdc[26436](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502429, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:40:29.800841 27355 init.cc:377] Logged in from keytab as kudu/127.7.132.130@KRBTEST.COM (short username kudu)
I20260812 02:40:29.802470 27355 webserver.cc:460] Webserver started at http://127.7.132.130:36611/ using document root <none> and password file <none>
I20260812 02:40:29.803220 27355 fs_manager.cc:359] Metadata directory not provided
I20260812 02:40:29.803376 27355 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:40:29.808890 27355 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 02:40:29.811816 27377 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:40:29.813233 27355 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 02:40:29.813422 27355 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-1/wal
uuid: "c2298009914a4733a6b0604dd0843833"
format_stamp: "Formatted at 2026-08-12 02:40:06 on dist-test-slave-mn4r"
server_key: "1b234fb35756c4ca2b9a5ad2ac41f3fd"
server_key_iv: "0df15e5f009641f6ebcb4ec45d2108ba"
server_key_version: "encryptionkey@0"
I20260812 02:40:29.814523 27355 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:40:29.829999 27355 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:40:29.834301 27355 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:40:29.834671 27355 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:40:29.835615 27355 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:40:29.836902 27355 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:40:29.836999 27355 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:40:29.837081 27355 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:40:29.837139 27355 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:40:29.872459 27355 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.130:37905
I20260812 02:40:29.872581 27505 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.130:37905 every 8 connection(s)
I20260812 02:40:29.874202 27355 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-1/data/info.pb
I20260812 02:40:29.874650 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 27355
I20260812 02:40:29.875391 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.131:34721
--local_ip_for_outbound_sockets=127.7.132.131
--tserver_master_addrs=127.7.132.190:42683
--webserver_port=40209
--webserver_interface=127.7.132.131
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.131.keytab
--principal=kudu/127.7.132.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:44005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 02:40:29 dist-test-slave-mn4r krb5kdc[26436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502429, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.130@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 02:40:29.911798 27353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:40:29.879465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:45533 (local address 127.7.132.190:42683)
0812 02:40:29.879691 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0812 02:40:29.879699 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:40:29.890981 (+ 11282us) server_negotiation.cc:407] Connection header received
0812 02:40:29.894046 (+ 3065us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:40:29.894053 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:40:29.894252 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:40:29.894384 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:40:29.896718 (+ 2334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:40:29.897998 (+ 1280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:40:29.900731 (+ 2733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:40:29.901195 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:40:29.901301 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:40:29.905880 (+ 4579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:40:29.905885 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:40:29.905889 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:40:29.905923 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:40:29.908541 (+ 2618us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:40:29.909334 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:40:29.909342 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:40:29.909345 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:40:29.909486 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:40:29.909942 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:40:29.909948 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:40:29.909951 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:40:29.910303 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:40:29.910462 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0812 02:40:29.911168 (+ 706us) server_negotiation.cc:299] Negotiation successful
0812 02:40:29.911403 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260812 02:40:29.912858 27508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:40:29.878735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:42683 (local address 127.7.132.130:45533)
0812 02:40:29.890731 (+ 11996us) negotiation.cc:107] Waiting for socket to connect
0812 02:40:29.890796 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0812 02:40:29.893682 (+ 2886us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:40:29.894718 (+ 1036us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:40:29.894733 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:40:29.895565 (+ 832us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:40:29.896458 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:40:29.896510 (+ 52us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:40:29.898233 (+ 1723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:40:29.898241 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:40:29.900549 (+ 2308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:40:29.900568 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:40:29.901445 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:40:29.901454 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:40:29.901807 (+ 353us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:40:29.902683 (+ 876us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:40:29.902720 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:40:29.905617 (+ 2897us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:40:29.908707 (+ 3090us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:40:29.908717 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:40:29.908735 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:40:29.909195 (+ 460us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:40:29.909612 (+ 417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:40:29.909616 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:40:29.909620 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:40:29.909822 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:40:29.910443 (+ 621us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:40:29.910452 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:40:29.910803 (+ 351us) client_negotiation.cc:769] Sending connection context
0812 02:40:29.911082 (+ 279us) client_negotiation.cc:240] Negotiation successful
0812 02:40:29.911684 (+ 602us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11607,"thread_start_us":11341,"threads_started":1}
I20260812 02:40:29.914487 27506 heartbeater.cc:347] Connected to a master server at 127.7.132.190:42683
I20260812 02:40:29.914810 27506 heartbeater.cc:464] Registering TS with master...
I20260812 02:40:29.915587 27506 heartbeater.cc:511] Master 127.7.132.190:42683 requested a full tablet report, sending...
I20260812 02:40:29.917121 27132 ts_manager.cc:194] Registered new tserver with Master: c2298009914a4733a6b0604dd0843833 (127.7.132.130:37905)
I20260812 02:40:29.917938 27132 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.130@KRBTEST.COM'} at 127.7.132.130:45533
W20260812 02:40:30.236320 27509 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:40:30.236708 27509 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:40:30.236912 27509 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:40:30.245827 27509 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:40:30.245996 27509 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:40:30.246225 27509 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.131
I20260812 02:40:30.257519 27509 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:44005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.131.keytab
--principal=kudu/127.7.132.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.131:34721
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.132.131
--webserver_port=40209
--tserver_master_addrs=127.7.132.190:42683
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.131
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:40:30.259017 27509 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:40:30.260540 27509 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:40:30.275723 27518 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:40:30.275679 27519 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:40:30.275781 27521 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:40:30.276664 27509 server_base.cc:1034] running on GCE node
I20260812 02:40:30.277331 27509 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:40:30.278448 27509 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:40:30.279731 27509 hybrid_clock.cc:648] HybridClock initialized: now 1786502430279667 us; error 62 us; skew 500 ppm
I20260812 02:40:30.280100 27509 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:40:30 dist-test-slave-mn4r krb5kdc[26436](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502430, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:40:30.283499 27509 init.cc:377] Logged in from keytab as kudu/127.7.132.131@KRBTEST.COM (short username kudu)
I20260812 02:40:30.285154 27509 webserver.cc:460] Webserver started at http://127.7.132.131:40209/ using document root <none> and password file <none>
I20260812 02:40:30.285852 27509 fs_manager.cc:359] Metadata directory not provided
I20260812 02:40:30.286029 27509 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:40:30.291556 27509 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.000s sys 0.004s
I20260812 02:40:30.294397 27528 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:40:30.295815 27509 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 02:40:30.296030 27509 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-2/wal
uuid: "6ae48edd11834d13a65cb21c37afb69a"
format_stamp: "Formatted at 2026-08-12 02:40:07 on dist-test-slave-mn4r"
server_key: "ffd82d769373b97c9960a53afc1bfa1e"
server_key_iv: "1ae78bcf9dea599eaaa8593e58113d0c"
server_key_version: "encryptionkey@0"
I20260812 02:40:30.297063 27509 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:40:30.327699 27509 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:40:30.332428 27509 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:40:30.332736 27509 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:40:30.333781 27509 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:40:30.335296 27509 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:40:30.335430 27509 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:40:30.335561 27509 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:40:30.335651 27509 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:40:30.372661 27509 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.131:34721
I20260812 02:40:30.372720 27658 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.131:34721 every 8 connection(s)
I20260812 02:40:30.374238 27509 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-2/data/info.pb
I20260812 02:40:30.383755 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 27509
Aug 12 02:40:30 dist-test-slave-mn4r krb5kdc[26436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502430, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.131@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 02:40:30.396206 27353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:40:30.376842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:57197 (local address 127.7.132.190:42683)
0812 02:40:30.377034 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0812 02:40:30.377039 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:40:30.378245 (+ 1206us) server_negotiation.cc:407] Connection header received
0812 02:40:30.380072 (+ 1827us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:40:30.380077 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:40:30.380146 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:40:30.380236 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:40:30.381723 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:40:30.382589 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:40:30.384240 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:40:30.384740 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:40:30.384879 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:40:30.389324 (+ 4445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:40:30.389329 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:40:30.389333 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:40:30.389365 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:40:30.392802 (+ 3437us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:40:30.393553 (+ 751us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:40:30.393559 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:40:30.393564 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:40:30.393729 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:40:30.394239 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:40:30.394246 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:40:30.394250 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:40:30.394622 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:40:30.394790 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0812 02:40:30.395610 (+ 820us) server_negotiation.cc:299] Negotiation successful
0812 02:40:30.395840 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 02:40:30.397609 27661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:40:30.377324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:42683 (local address 127.7.132.131:57197)
0812 02:40:30.378022 (+ 698us) negotiation.cc:107] Waiting for socket to connect
0812 02:40:30.378073 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 02:40:30.379807 (+ 1734us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:40:30.380457 (+ 650us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:40:30.380465 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:40:30.380973 (+ 508us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:40:30.381555 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:40:30.381585 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:40:30.382747 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:40:30.382750 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:40:30.384078 (+ 1328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:40:30.384092 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:40:30.384991 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:40:30.385000 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:40:30.385332 (+ 332us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:40:30.386149 (+ 817us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:40:30.386185 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:40:30.389115 (+ 2930us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:40:30.392946 (+ 3831us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:40:30.392955 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:40:30.392969 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:40:30.393399 (+ 430us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:40:30.393853 (+ 454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:40:30.393856 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:40:30.393858 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:40:30.394061 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:40:30.394768 (+ 707us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:40:30.394774 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:40:30.395164 (+ 390us) client_negotiation.cc:769] Sending connection context
0812 02:40:30.395501 (+ 337us) client_negotiation.cc:240] Negotiation successful
0812 02:40:30.396088 (+ 587us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":277,"threads_started":1}
I20260812 02:40:30.401660 27659 heartbeater.cc:347] Connected to a master server at 127.7.132.190:42683
I20260812 02:40:30.401803 27663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:40:30.384794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37420 (local address 127.7.132.190:42683)
0812 02:40:30.385814 (+ 1020us) server_negotiation.cc:206] Beginning negotiation
0812 02:40:30.385819 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:40:30.385834 (+ 15us) server_negotiation.cc:407] Connection header received
0812 02:40:30.386275 (+ 441us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:40:30.386279 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:40:30.386339 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:40:30.386482 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:40:30.388602 (+ 2120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:40:30.389606 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:40:30.391226 (+ 1620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:40:30.391698 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:40:30.391795 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:40:30.395118 (+ 3323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:40:30.395125 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:40:30.395129 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:40:30.395161 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:40:30.398488 (+ 3327us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:40:30.399368 (+ 880us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:40:30.399372 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:40:30.399375 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:40:30.399516 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:40:30.400002 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:40:30.400005 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:40:30.400007 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:40:30.400266 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:40:30.400551 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0812 02:40:30.401101 (+ 550us) server_negotiation.cc:299] Negotiation successful
0812 02:40:30.401377 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":935,"thread_start_us":815,"threads_started":1}
I20260812 02:40:30.402041 27659 heartbeater.cc:464] Registering TS with master...
I20260812 02:40:30.402979 27659 heartbeater.cc:511] Master 127.7.132.190:42683 requested a full tablet report, sending...
I20260812 02:40:30.404580 27131 ts_manager.cc:194] Registered new tserver with Master: 6ae48edd11834d13a65cb21c37afb69a (127.7.132.131:34721)
I20260812 02:40:30.405443 27131 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.131@KRBTEST.COM'} at 127.7.132.131:57197
I20260812 02:40:30.417604 7698 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 02:40:30.425211 27663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:40:30.418896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37432 (local address 127.7.132.190:42683)
0812 02:40:30.419148 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0812 02:40:30.419156 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:40:30.419577 (+ 421us) server_negotiation.cc:407] Connection header received
0812 02:40:30.419876 (+ 299us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:40:30.419880 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:40:30.420024 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:40:30.420190 (+ 166us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 02:40:30.420651 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:40:30.421394 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:40:30.423026 (+ 1632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:40:30.423522 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:40:30.423668 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:40:30.424202 (+ 534us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 02:40:30.424427 (+ 225us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0812 02:40:30.424896 (+ 469us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:37432: token expired
Metrics: {"server-negotiator.queue_time_us":65}
W20260812 02:40:30.425330 27663 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:37432: token expired
W20260812 02:40:30.426028 27031 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.7.132.190:42683)
I20260812 02:40:30.438585 27350 heartbeater.cc:503] Master 127.7.132.190:42683 was elected leader, sending a full tablet report...
I20260812 02:40:30.440979 27663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:40:30.426699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37440 (local address 127.7.132.190:42683)
0812 02:40:30.426878 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0812 02:40:30.426886 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:40:30.426906 (+ 20us) server_negotiation.cc:407] Connection header received
0812 02:40:30.427132 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:40:30.427137 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:40:30.427205 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:40:30.427306 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:40:30.428767 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:40:30.429543 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:40:30.431065 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:40:30.431481 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:40:30.431628 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:40:30.434562 (+ 2934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:40:30.434567 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:40:30.434570 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:40:30.434604 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:40:30.437228 (+ 2624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:40:30.438440 (+ 1212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:40:30.438445 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:40:30.438447 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:40:30.438610 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:40:30.439159 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:40:30.439171 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:40:30.439173 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:40:30.439636 (+ 463us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:40:30.439927 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0812 02:40:30.440349 (+ 422us) server_negotiation.cc:299] Negotiation successful
0812 02:40:30.440622 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 02:40:30.444550 27131 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37440:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 02:40:30.451995 27131 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 02:40:30.489261 27669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:40:30.464352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.130:37905 (local address 127.0.0.1:60134)
0812 02:40:30.465340 (+ 988us) negotiation.cc:107] Waiting for socket to connect
0812 02:40:30.465388 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 02:40:30.465566 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:40:30.481455 (+ 15889us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:40:30.481465 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:40:30.481495 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:40:30.482181 (+ 686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:40:30.482204 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:40:30.483878 (+ 1674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:40:30.483885 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:40:30.485985 (+ 2100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:40:30.486002 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:40:30.488092 (+ 2090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:40:30.488099 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:40:30.488311 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:40:30.488373 (+ 62us) client_negotiation.cc:769] Sending connection context
0812 02:40:30.488527 (+ 154us) client_negotiation.cc:240] Negotiation successful
0812 02:40:30.488784 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":735,"thread_start_us":514,"threads_started":1}
I20260812 02:40:30.489903 27674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:40:30.466822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41614 (local address 127.7.132.131:34721)
0812 02:40:30.480256 (+ 13434us) server_negotiation.cc:206] Beginning negotiation
0812 02:40:30.480269 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 02:40:30.480986 (+ 717us) server_negotiation.cc:407] Connection header received
0812 02:40:30.481106 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:40:30.481112 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:40:30.481292 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:40:30.481507 (+ 215us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:40:30.482785 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:40:30.484675 (+ 1890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:40:30.486975 (+ 2300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:40:30.488199 (+ 1224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:40:30.488327 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:40:30.488560 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0812 02:40:30.489384 (+ 824us) server_negotiation.cc:299] Negotiation successful
0812 02:40:30.489561 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13175,"thread_start_us":12959,"threads_started":1}
I20260812 02:40:30.489970 27673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:40:30.466468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.131:34721 (local address 127.0.0.1:41614)
0812 02:40:30.480760 (+ 14292us) negotiation.cc:107] Waiting for socket to connect
0812 02:40:30.480782 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 02:40:30.480923 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:40:30.481719 (+ 796us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:40:30.481725 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:40:30.481775 (+ 50us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:40:30.482487 (+ 712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:40:30.482513 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:40:30.484826 (+ 2313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:40:30.484832 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:40:30.486810 (+ 1978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:40:30.486823 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:40:30.488713 (+ 1890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:40:30.488720 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:40:30.488930 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:40:30.488968 (+ 38us) client_negotiation.cc:769] Sending connection context
0812 02:40:30.489051 (+ 83us) client_negotiation.cc:240] Negotiation successful
0812 02:40:30.489216 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14142,"mutex_wait_us":76,"spinlock_wait_cycles":7808,"thread_start_us":14007,"threads_started":1}
I20260812 02:40:30.490509 27672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:40:30.465021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60134 (local address 127.7.132.130:37905)
0812 02:40:30.480631 (+ 15610us) server_negotiation.cc:206] Beginning negotiation
0812 02:40:30.480642 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 02:40:30.480682 (+ 40us) server_negotiation.cc:407] Connection header received
0812 02:40:30.480813 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:40:30.480824 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:40:30.481161 (+ 337us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:40:30.481408 (+ 247us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:40:30.482442 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:40:30.483737 (+ 1295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:40:30.486211 (+ 2474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:40:30.487841 (+ 1630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:40:30.488025 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:40:30.489860 (+ 1835us) server_negotiation.cc:1035] Waiting for connection context
0812 02:40:30.489947 (+ 87us) server_negotiation.cc:299] Negotiation successful
0812 02:40:30.490174 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15398,"thread_start_us":15280,"threads_started":1}
I20260812 02:40:30.490535 27670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:40:30.464352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:41881 (local address 127.0.0.1:59444)
0812 02:40:30.466270 (+ 1918us) negotiation.cc:107] Waiting for socket to connect
0812 02:40:30.466291 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 02:40:30.466416 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:40:30.481719 (+ 15303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:40:30.481725 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:40:30.481767 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:40:30.482586 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:40:30.482600 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:40:30.485106 (+ 2506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:40:30.485114 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:40:30.487410 (+ 2296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:40:30.487427 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:40:30.489310 (+ 1883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:40:30.489320 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:40:30.489803 (+ 483us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:40:30.489839 (+ 36us) client_negotiation.cc:769] Sending connection context
0812 02:40:30.489921 (+ 82us) client_negotiation.cc:240] Negotiation successful
0812 02:40:30.490285 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1766,"mutex_wait_us":113,"spinlock_wait_cycles":1280,"thread_start_us":1507,"threads_started":1}
I20260812 02:40:30.491472 27671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:40:30.464684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59444 (local address 127.7.132.129:41881)
0812 02:40:30.481038 (+ 16354us) server_negotiation.cc:206] Beginning negotiation
0812 02:40:30.481047 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 02:40:30.481075 (+ 28us) server_negotiation.cc:407] Connection header received
0812 02:40:30.481176 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:40:30.481185 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:40:30.481453 (+ 268us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:40:30.481857 (+ 404us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:40:30.483166 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:40:30.484958 (+ 1792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:40:30.487615 (+ 2657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:40:30.489073 (+ 1458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:40:30.490755 (+ 1682us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:40:30.490964 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0812 02:40:30.491033 (+ 69us) server_negotiation.cc:299] Negotiation successful
0812 02:40:30.491151 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16164,"thread_start_us":16017,"threads_started":1}
I20260812 02:40:30.494216 27428 tablet_service.cc:1467] Processing CreateTablet for tablet c991e46989b24e1db557b54e0a2501b3 (DEFAULT_TABLE table=test-table [id=79f935ef57a44430bd9f2859a223e762]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:40:30.494216 27585 tablet_service.cc:1467] Processing CreateTablet for tablet c991e46989b24e1db557b54e0a2501b3 (DEFAULT_TABLE table=test-table [id=79f935ef57a44430bd9f2859a223e762]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:40:30.495661 27428 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c991e46989b24e1db557b54e0a2501b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:40:30.495708 27585 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c991e46989b24e1db557b54e0a2501b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:40:30.495306 27273 tablet_service.cc:1467] Processing CreateTablet for tablet c991e46989b24e1db557b54e0a2501b3 (DEFAULT_TABLE table=test-table [id=79f935ef57a44430bd9f2859a223e762]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:40:30.496667 27273 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c991e46989b24e1db557b54e0a2501b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:40:30.510921 27676 tablet_bootstrap.cc:492] T c991e46989b24e1db557b54e0a2501b3 P c2298009914a4733a6b0604dd0843833: Bootstrap starting.
I20260812 02:40:30.512454 27677 tablet_bootstrap.cc:492] T c991e46989b24e1db557b54e0a2501b3 P 6ae48edd11834d13a65cb21c37afb69a: Bootstrap starting.
I20260812 02:40:30.516287 27676 tablet_bootstrap.cc:654] T c991e46989b24e1db557b54e0a2501b3 P c2298009914a4733a6b0604dd0843833: Neither blocks nor log segments found. Creating new log.
I20260812 02:40:30.516458 27677 tablet_bootstrap.cc:654] T c991e46989b24e1db557b54e0a2501b3 P 6ae48edd11834d13a65cb21c37afb69a: Neither blocks nor log segments found. Creating new log.
I20260812 02:40:30.517216 27678 tablet_bootstrap.cc:492] T c991e46989b24e1db557b54e0a2501b3 P cabf179ab4374509b0163a571e20f483: Bootstrap starting.
I20260812 02:40:30.518112 27677 log.cc:824] T c991e46989b24e1db557b54e0a2501b3 P 6ae48edd11834d13a65cb21c37afb69a: Log is configured to *not* fsync() on all Append() calls
I20260812 02:40:30.519177 27676 log.cc:824] T c991e46989b24e1db557b54e0a2501b3 P c2298009914a4733a6b0604dd0843833: Log is configured to *not* fsync() on all Append() calls
I20260812 02:40:30.520839 27678 tablet_bootstrap.cc:654] T c991e46989b24e1db557b54e0a2501b3 P cabf179ab4374509b0163a571e20f483: Neither blocks nor log segments found. Creating new log.
I20260812 02:40:30.521812 27677 tablet_bootstrap.cc:492] T c991e46989b24e1db557b54e0a2501b3 P 6ae48edd11834d13a65cb21c37afb69a: No bootstrap required, opened a new log
I20260812 02:40:30.522169 27677 ts_tablet_manager.cc:1397] T c991e46989b24e1db557b54e0a2501b3 P 6ae48edd11834d13a65cb21c37afb69a: Time spent bootstrapping tablet: real 0.010s user 0.009s sys 0.000s
I20260812 02:40:30.522259 27678 log.cc:824] T c991e46989b24e1db557b54e0a2501b3 P cabf179ab4374509b0163a571e20f483: Log is configured to *not* fsync() on all Append() calls
I20260812 02:40:30.522670 27676 tablet_bootstrap.cc:492] T c991e46989b24e1db557b54e0a2501b3 P c2298009914a4733a6b0604dd0843833: No bootstrap required, opened a new log
I20260812 02:40:30.522984 27676 ts_tablet_manager.cc:1397] T c991e46989b24e1db557b54e0a2501b3 P c2298009914a4733a6b0604dd0843833: Time spent bootstrapping tablet: real 0.012s user 0.009s sys 0.000s
I20260812 02:40:30.525530 27678 tablet_bootstrap.cc:492] T c991e46989b24e1db557b54e0a2501b3 P cabf179ab4374509b0163a571e20f483: No bootstrap required, opened a new log
I20260812 02:40:30.525797 27678 ts_tablet_manager.cc:1397] T c991e46989b24e1db557b54e0a2501b3 P cabf179ab4374509b0163a571e20f483: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260812 02:40:30.532663 27676 raft_consensus.cc:348] T c991e46989b24e1db557b54e0a2501b3 P c2298009914a4733a6b0604dd0843833 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cabf179ab4374509b0163a571e20f483" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 41881 } } peers { permanent_uuid: "c2298009914a4733a6b0604dd0843833" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37905 } } peers { permanent_uuid: "6ae48edd11834d13a65cb21c37afb69a" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 34721 } }
I20260812 02:40:30.533092 27676 raft_consensus.cc:374] T c991e46989b24e1db557b54e0a2501b3 P c2298009914a4733a6b0604dd0843833 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:40:30.533209 27676 raft_consensus.cc:729] T c991e46989b24e1db557b54e0a2501b3 P c2298009914a4733a6b0604dd0843833 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2298009914a4733a6b0604dd0843833, State: Initialized, Role: FOLLOWER
I20260812 02:40:30.533058 27677 raft_consensus.cc:348] T c991e46989b24e1db557b54e0a2501b3 P 6ae48edd11834d13a65cb21c37afb69a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cabf179ab4374509b0163a571e20f483" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 41881 } } peers { permanent_uuid: "c2298009914a4733a6b0604dd0843833" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37905 } } peers { permanent_uuid: "6ae48edd11834d13a65cb21c37afb69a" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 34721 } }
I20260812 02:40:30.533504 27677 raft_consensus.cc:374] T c991e46989b24e1db557b54e0a2501b3 P 6ae48edd11834d13a65cb21c37afb69a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:40:30.533645 27677 raft_consensus.cc:729] T c991e46989b24e1db557b54e0a2501b3 P 6ae48edd11834d13a65cb21c37afb69a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ae48edd11834d13a65cb21c37afb69a, State: Initialized, Role: FOLLOWER
I20260812 02:40:30.533957 27676 consensus_queue.cc:260] T c991e46989b24e1db557b54e0a2501b3 P c2298009914a4733a6b0604dd0843833 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cabf179ab4374509b0163a571e20f483" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 41881 } } peers { permanent_uuid: "c2298009914a4733a6b0604dd0843833" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37905 } } peers { permanent_uuid: "6ae48edd11834d13a65cb21c37afb69a" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 34721 } }
I20260812 02:40:30.534263 27677 consensus_queue.cc:260] T c991e46989b24e1db557b54e0a2501b3 P 6ae48edd11834d13a65cb21c37afb69a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cabf179ab4374509b0163a571e20f483" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 41881 } } peers { permanent_uuid: "c2298009914a4733a6b0604dd0843833" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37905 } } peers { permanent_uuid: "6ae48edd11834d13a65cb21c37afb69a" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 34721 } }
I20260812 02:40:30.535342 27506 heartbeater.cc:503] Master 127.7.132.190:42683 was elected leader, sending a full tablet report...
I20260812 02:40:30.536121 27676 ts_tablet_manager.cc:1428] T c991e46989b24e1db557b54e0a2501b3 P c2298009914a4733a6b0604dd0843833: Time spent starting tablet: real 0.013s user 0.011s sys 0.000s
I20260812 02:40:30.536845 27678 raft_consensus.cc:348] T c991e46989b24e1db557b54e0a2501b3 P cabf179ab4374509b0163a571e20f483 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cabf179ab4374509b0163a571e20f483" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 41881 } } peers { permanent_uuid: "c2298009914a4733a6b0604dd0843833" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37905 } } peers { permanent_uuid: "6ae48edd11834d13a65cb21c37afb69a" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 34721 } }
I20260812 02:40:30.537185 27659 heartbeater.cc:503] Master 127.7.132.190:42683 was elected leader, sending a full tablet report...
I20260812 02:40:30.537293 27678 raft_consensus.cc:374] T c991e46989b24e1db557b54e0a2501b3 P cabf179ab4374509b0163a571e20f483 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:40:30.537405 27678 raft_consensus.cc:729] T c991e46989b24e1db557b54e0a2501b3 P cabf179ab4374509b0163a571e20f483 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cabf179ab4374509b0163a571e20f483, State: Initialized, Role: FOLLOWER
I20260812 02:40:30.537808 27677 ts_tablet_manager.cc:1428] T c991e46989b24e1db557b54e0a2501b3 P 6ae48edd11834d13a65cb21c37afb69a: Time spent starting tablet: real 0.016s user 0.010s sys 0.004s
I20260812 02:40:30.538008 27678 consensus_queue.cc:260] T c991e46989b24e1db557b54e0a2501b3 P cabf179ab4374509b0163a571e20f483 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cabf179ab4374509b0163a571e20f483" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 41881 } } peers { permanent_uuid: "c2298009914a4733a6b0604dd0843833" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37905 } } peers { permanent_uuid: "6ae48edd11834d13a65cb21c37afb69a" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 34721 } }
I20260812 02:40:30.539695 27678 ts_tablet_manager.cc:1428] T c991e46989b24e1db557b54e0a2501b3 P cabf179ab4374509b0163a571e20f483: Time spent starting tablet: real 0.014s user 0.011s sys 0.004s
W20260812 02:40:30.626008 27660 tablet.cc:2369] T c991e46989b24e1db557b54e0a2501b3 P 6ae48edd11834d13a65cb21c37afb69a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 02:40:30.629861 27683 raft_consensus.cc:482] T c991e46989b24e1db557b54e0a2501b3 P 6ae48edd11834d13a65cb21c37afb69a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 02:40:30.630244 27683 raft_consensus.cc:504] T c991e46989b24e1db557b54e0a2501b3 P 6ae48edd11834d13a65cb21c37afb69a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cabf179ab4374509b0163a571e20f483" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 41881 } } peers { permanent_uuid: "c2298009914a4733a6b0604dd0843833" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37905 } } peers { permanent_uuid: "6ae48edd11834d13a65cb21c37afb69a" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 34721 } }
I20260812 02:40:30.631984 27683 leader_election.cc:288] T c991e46989b24e1db557b54e0a2501b3 P 6ae48edd11834d13a65cb21c37afb69a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cabf179ab4374509b0163a571e20f483 (127.7.132.129:41881), c2298009914a4733a6b0604dd0843833 (127.7.132.130:37905)
I20260812 02:40:30.638700 27671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:40:30.632655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:37713 (local address 127.7.132.129:41881)
0812 02:40:30.632853 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0812 02:40:30.632861 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:40:30.632885 (+ 24us) server_negotiation.cc:407] Connection header received
0812 02:40:30.633103 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:40:30.633108 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:40:30.633195 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:40:30.633309 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:40:30.633889 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:40:30.634889 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:40:30.636762 (+ 1873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:40:30.637841 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:40:30.637965 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:40:30.638017 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0812 02:40:30.638409 (+ 392us) server_negotiation.cc:299] Negotiation successful
0812 02:40:30.638516 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260812 02:40:30.638926 27661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:40:30.632351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:41881 (local address 127.7.132.131:37713)
0812 02:40:30.632720 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0812 02:40:30.632747 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0812 02:40:30.632908 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:40:30.633360 (+ 452us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:40:30.633364 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:40:30.633395 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:40:30.633705 (+ 310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:40:30.633717 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:40:30.635023 (+ 1306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:40:30.635029 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:40:30.636596 (+ 1567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:40:30.636609 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:40:30.637993 (+ 1384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:40:30.637999 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:40:30.638233 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:40:30.638277 (+ 44us) client_negotiation.cc:769] Sending connection context
0812 02:40:30.638392 (+ 115us) client_negotiation.cc:240] Negotiation successful
0812 02:40:30.638534 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195,"mutex_wait_us":1,"spinlock_wait_cycles":25984}
I20260812 02:40:30.639735 27672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:40:30.632552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:58963 (local address 127.7.132.130:37905)
0812 02:40:30.632822 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0812 02:40:30.632828 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:40:30.633472 (+ 644us) server_negotiation.cc:407] Connection header received
0812 02:40:30.633664 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:40:30.633670 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:40:30.633747 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:40:30.633843 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:40:30.634541 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:40:30.635486 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:40:30.637582 (+ 2096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:40:30.638685 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:40:30.638842 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:40:30.638987 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0812 02:40:30.639352 (+ 365us) server_negotiation.cc:299] Negotiation successful
0812 02:40:30.639496 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124}
I20260812 02:40:30.639865 27685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:40:30.632434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.130:37905 (local address 127.7.132.131:58963)
0812 02:40:30.633352 (+ 918us) negotiation.cc:107] Waiting for socket to connect
0812 02:40:30.633370 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 02:40:30.633513 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:40:30.633876 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:40:30.633880 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:40:30.633903 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:40:30.634370 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:40:30.634381 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:40:30.635641 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:40:30.635645 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:40:30.637413 (+ 1768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:40:30.637428 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:40:30.638937 (+ 1509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:40:30.638944 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:40:30.639169 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:40:30.639206 (+ 37us) client_negotiation.cc:769] Sending connection context
0812 02:40:30.639318 (+ 112us) client_negotiation.cc:240] Negotiation successful
0812 02:40:30.639460 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":792,"mutex_wait_us":46,"thread_start_us":618,"threads_started":1}
I20260812 02:40:30.640368 27299 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c991e46989b24e1db557b54e0a2501b3" candidate_uuid: "6ae48edd11834d13a65cb21c37afb69a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cabf179ab4374509b0163a571e20f483" is_pre_election: true
W20260812 02:40:30.640662 27507 tablet.cc:2369] T c991e46989b24e1db557b54e0a2501b3 P c2298009914a4733a6b0604dd0843833: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 02:40:30.641067 27299 raft_consensus.cc:2454] T c991e46989b24e1db557b54e0a2501b3 P cabf179ab4374509b0163a571e20f483 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ae48edd11834d13a65cb21c37afb69a in term 0.
I20260812 02:40:30.641356 27453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c991e46989b24e1db557b54e0a2501b3" candidate_uuid: "6ae48edd11834d13a65cb21c37afb69a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2298009914a4733a6b0604dd0843833" is_pre_election: true
I20260812 02:40:30.642019 27453 raft_consensus.cc:2454] T c991e46989b24e1db557b54e0a2501b3 P c2298009914a4733a6b0604dd0843833 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ae48edd11834d13a65cb21c37afb69a in term 0.
I20260812 02:40:30.641978 27529 leader_election.cc:302] T c991e46989b24e1db557b54e0a2501b3 P 6ae48edd11834d13a65cb21c37afb69a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ae48edd11834d13a65cb21c37afb69a, cabf179ab4374509b0163a571e20f483; no voters:
I20260812 02:40:30.642477 27683 raft_consensus.cc:2781] T c991e46989b24e1db557b54e0a2501b3 P 6ae48edd11834d13a65cb21c37afb69a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 02:40:30.642658 27683 raft_consensus.cc:482] T c991e46989b24e1db557b54e0a2501b3 P 6ae48edd11834d13a65cb21c37afb69a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 02:40:30.642772 27683 raft_consensus.cc:3037] T c991e46989b24e1db557b54e0a2501b3 P 6ae48edd11834d13a65cb21c37afb69a [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:40:30.645212 27683 raft_consensus.cc:504] T c991e46989b24e1db557b54e0a2501b3 P 6ae48edd11834d13a65cb21c37afb69a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cabf179ab4374509b0163a571e20f483" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 41881 } } peers { permanent_uuid: "c2298009914a4733a6b0604dd0843833" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37905 } } peers { permanent_uuid: "6ae48edd11834d13a65cb21c37afb69a" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 34721 } }
I20260812 02:40:30.645850 27683 leader_election.cc:288] T c991e46989b24e1db557b54e0a2501b3 P 6ae48edd11834d13a65cb21c37afb69a [CANDIDATE]: Term 1 election: Requested vote from peers cabf179ab4374509b0163a571e20f483 (127.7.132.129:41881), c2298009914a4733a6b0604dd0843833 (127.7.132.130:37905)
I20260812 02:40:30.646468 27299 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c991e46989b24e1db557b54e0a2501b3" candidate_uuid: "6ae48edd11834d13a65cb21c37afb69a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cabf179ab4374509b0163a571e20f483"
I20260812 02:40:30.646504 27453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c991e46989b24e1db557b54e0a2501b3" candidate_uuid: "6ae48edd11834d13a65cb21c37afb69a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2298009914a4733a6b0604dd0843833"
I20260812 02:40:30.646687 27299 raft_consensus.cc:3037] T c991e46989b24e1db557b54e0a2501b3 P cabf179ab4374509b0163a571e20f483 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:40:30.646727 27453 raft_consensus.cc:3037] T c991e46989b24e1db557b54e0a2501b3 P c2298009914a4733a6b0604dd0843833 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:40:30.648900 27299 raft_consensus.cc:2454] T c991e46989b24e1db557b54e0a2501b3 P cabf179ab4374509b0163a571e20f483 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ae48edd11834d13a65cb21c37afb69a in term 1.
I20260812 02:40:30.649235 27453 raft_consensus.cc:2454] T c991e46989b24e1db557b54e0a2501b3 P c2298009914a4733a6b0604dd0843833 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ae48edd11834d13a65cb21c37afb69a in term 1.
I20260812 02:40:30.649320 27529 leader_election.cc:302] T c991e46989b24e1db557b54e0a2501b3 P 6ae48edd11834d13a65cb21c37afb69a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ae48edd11834d13a65cb21c37afb69a, cabf179ab4374509b0163a571e20f483; no voters:
I20260812 02:40:30.649936 27683 raft_consensus.cc:2781] T c991e46989b24e1db557b54e0a2501b3 P 6ae48edd11834d13a65cb21c37afb69a [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:40:30.650437 27683 raft_consensus.cc:686] T c991e46989b24e1db557b54e0a2501b3 P 6ae48edd11834d13a65cb21c37afb69a [term 1 LEADER]: Becoming Leader. State: Replica: 6ae48edd11834d13a65cb21c37afb69a, State: Running, Role: LEADER
I20260812 02:40:30.651007 27683 consensus_queue.cc:237] T c991e46989b24e1db557b54e0a2501b3 P 6ae48edd11834d13a65cb21c37afb69a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cabf179ab4374509b0163a571e20f483" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 41881 } } peers { permanent_uuid: "c2298009914a4733a6b0604dd0843833" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37905 } } peers { permanent_uuid: "6ae48edd11834d13a65cb21c37afb69a" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 34721 } }
W20260812 02:40:30.652643 27351 tablet.cc:2369] T c991e46989b24e1db557b54e0a2501b3 P cabf179ab4374509b0163a571e20f483: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 02:40:30.656229 27131 catalog_manager.cc:5495] T c991e46989b24e1db557b54e0a2501b3 P 6ae48edd11834d13a65cb21c37afb69a reported cstate change: term changed from 0 to 1, leader changed from <none> to 6ae48edd11834d13a65cb21c37afb69a (127.7.132.131). New cstate: current_term: 1 leader_uuid: "6ae48edd11834d13a65cb21c37afb69a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cabf179ab4374509b0163a571e20f483" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 41881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2298009914a4733a6b0604dd0843833" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37905 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ae48edd11834d13a65cb21c37afb69a" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 34721 } health_report { overall_health: HEALTHY } } }
I20260812 02:40:30.717093 27674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:40:30.711724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41616 (local address 127.7.132.131:34721)
0812 02:40:30.711975 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0812 02:40:30.711979 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 02:40:30.711995 (+ 16us) server_negotiation.cc:407] Connection header received
0812 02:40:30.712090 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:40:30.712093 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:40:30.712169 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:40:30.712332 (+ 163us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 02:40:30.712705 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:40:30.713417 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:40:30.714893 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:40:30.715315 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:40:30.715459 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:40:30.715964 (+ 505us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 02:40:30.716472 (+ 508us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 02:40:30.716637 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0812 02:40:30.716737 (+ 100us) server_negotiation.cc:299] Negotiation successful
0812 02:40:30.716823 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260812 02:40:30.725684 27299 raft_consensus.cc:1261] T c991e46989b24e1db557b54e0a2501b3 P cabf179ab4374509b0163a571e20f483 [term 1 FOLLOWER]: Refusing update from remote peer 6ae48edd11834d13a65cb21c37afb69a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:40:30.725692 27453 raft_consensus.cc:1261] T c991e46989b24e1db557b54e0a2501b3 P c2298009914a4733a6b0604dd0843833 [term 1 FOLLOWER]: Refusing update from remote peer 6ae48edd11834d13a65cb21c37afb69a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:40:30.726735 27683 consensus_queue.cc:1035] T c991e46989b24e1db557b54e0a2501b3 P 6ae48edd11834d13a65cb21c37afb69a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2298009914a4733a6b0604dd0843833" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37905 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:40:30.727035 27686 consensus_queue.cc:1035] T c991e46989b24e1db557b54e0a2501b3 P 6ae48edd11834d13a65cb21c37afb69a [LEADER]: Connected to new peer: Peer: permanent_uuid: "cabf179ab4374509b0163a571e20f483" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 41881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:40:30.755342 27132 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37440:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 02:40:30.755656 27132 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37440:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 02:40:30.759186 27132 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0fad46d3a46c4597903b423dfd124d70: Sending DeleteTablet for 3 replicas of tablet c991e46989b24e1db557b54e0a2501b3
I20260812 02:40:30.760479 27428 tablet_service.cc:1514] Processing DeleteTablet for tablet c991e46989b24e1db557b54e0a2501b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:40:30 UTC) from {username='kudu', principal='kudu/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:60134
I20260812 02:40:30.760558 27273 tablet_service.cc:1514] Processing DeleteTablet for tablet c991e46989b24e1db557b54e0a2501b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:40:30 UTC) from {username='kudu', principal='kudu/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:59444
I20260812 02:40:30.760551 27585 tablet_service.cc:1514] Processing DeleteTablet for tablet c991e46989b24e1db557b54e0a2501b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:40:30 UTC) from {username='kudu', principal='kudu/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:41614
I20260812 02:40:30.762138 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 27179
I20260812 02:40:30.773172 27700 tablet_replica.cc:330] T c991e46989b24e1db557b54e0a2501b3 P c2298009914a4733a6b0604dd0843833: stopping tablet replica
I20260812 02:40:30.773986 27700 raft_consensus.cc:2229] T c991e46989b24e1db557b54e0a2501b3 P c2298009914a4733a6b0604dd0843833 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 02:40:30.774874 27700 raft_consensus.cc:2258] T c991e46989b24e1db557b54e0a2501b3 P c2298009914a4733a6b0604dd0843833 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:40:30.778189 27700 ts_tablet_manager.cc:1905] T c991e46989b24e1db557b54e0a2501b3 P c2298009914a4733a6b0604dd0843833: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:40:30.779637 27702 tablet_replica.cc:330] T c991e46989b24e1db557b54e0a2501b3 P 6ae48edd11834d13a65cb21c37afb69a: stopping tablet replica
I20260812 02:40:30.780117 27702 raft_consensus.cc:2229] T c991e46989b24e1db557b54e0a2501b3 P 6ae48edd11834d13a65cb21c37afb69a [term 1 LEADER]: Raft consensus shutting down.
I20260812 02:40:30.780813 27702 raft_consensus.cc:2258] T c991e46989b24e1db557b54e0a2501b3 P 6ae48edd11834d13a65cb21c37afb69a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:40:30.783124 27702 ts_tablet_manager.cc:1905] T c991e46989b24e1db557b54e0a2501b3 P 6ae48edd11834d13a65cb21c37afb69a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:40:30.785677 27671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:40:30.763839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59454 (local address 127.7.132.129:41881)
0812 02:40:30.764014 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0812 02:40:30.764020 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:40:30.764052 (+ 32us) server_negotiation.cc:407] Connection header received
0812 02:40:30.764110 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:40:30.764115 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:40:30.764191 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:40:30.764290 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:40:30.766323 (+ 2033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:40:30.767575 (+ 1252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:40:30.769588 (+ 2013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:40:30.770334 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:40:30.770463 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:40:30.775823 (+ 5360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:40:30.775843 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:40:30.775861 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:40:30.775909 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:40:30.780650 (+ 4741us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:40:30.781842 (+ 1192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:40:30.781852 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:40:30.781855 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:40:30.782122 (+ 267us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:40:30.782650 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:40:30.782656 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:40:30.782660 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:40:30.783167 (+ 507us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:40:30.783574 (+ 407us) server_negotiation.cc:1035] Waiting for connection context
0812 02:40:30.785003 (+ 1429us) server_negotiation.cc:299] Negotiation successful
0812 02:40:30.785251 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260812 02:40:30.779063 27701 tablet_replica.cc:330] T c991e46989b24e1db557b54e0a2501b3 P cabf179ab4374509b0163a571e20f483: stopping tablet replica
I20260812 02:40:30.786582 27701 raft_consensus.cc:2229] T c991e46989b24e1db557b54e0a2501b3 P cabf179ab4374509b0163a571e20f483 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 02:40:30.786739 27343 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:40:30.787313 27700 ts_tablet_manager.cc:1918] T c991e46989b24e1db557b54e0a2501b3 P c2298009914a4733a6b0604dd0843833: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 02:40:30.787513 27700 log.cc:1196] T c991e46989b24e1db557b54e0a2501b3 P c2298009914a4733a6b0604dd0843833: Deleting WAL directory at /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-1/wal/wals/c991e46989b24e1db557b54e0a2501b3
I20260812 02:40:30.788130 27700 ts_tablet_manager.cc:1939] T c991e46989b24e1db557b54e0a2501b3 P c2298009914a4733a6b0604dd0843833: Deleting consensus metadata
I20260812 02:40:30.789858 27118 catalog_manager.cc:4841] TS c2298009914a4733a6b0604dd0843833 (127.7.132.130:37905): tablet c991e46989b24e1db557b54e0a2501b3 (table test-table [id=79f935ef57a44430bd9f2859a223e762]) successfully deleted
I20260812 02:40:30.794049 27702 ts_tablet_manager.cc:1918] T c991e46989b24e1db557b54e0a2501b3 P 6ae48edd11834d13a65cb21c37afb69a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 02:40:30.794368 27702 log.cc:1196] T c991e46989b24e1db557b54e0a2501b3 P 6ae48edd11834d13a65cb21c37afb69a: Deleting WAL directory at /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-2/wal/wals/c991e46989b24e1db557b54e0a2501b3
I20260812 02:40:30.795045 27702 ts_tablet_manager.cc:1939] T c991e46989b24e1db557b54e0a2501b3 P 6ae48edd11834d13a65cb21c37afb69a: Deleting consensus metadata
I20260812 02:40:30.797161 27115 catalog_manager.cc:4841] TS 6ae48edd11834d13a65cb21c37afb69a (127.7.132.131:34721): tablet c991e46989b24e1db557b54e0a2501b3 (table test-table [id=79f935ef57a44430bd9f2859a223e762]) successfully deleted
I20260812 02:40:30.893662 27701 raft_consensus.cc:2258] T c991e46989b24e1db557b54e0a2501b3 P cabf179ab4374509b0163a571e20f483 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:40:30.894203 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 27179
W20260812 02:40:30.921809 27115 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.132.129:41881 (error 108)
I20260812 02:40:30.922142 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 27355
I20260812 02:40:30.924113 27673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:40:30.923613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:41881 (local address 127.0.0.1:59456)
0812 02:40:30.923788 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0812 02:40:30.923916 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.129:41881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260812 02:40:30.924427 27115 catalog_manager.cc:4584] TS cabf179ab4374509b0163a571e20f483 (127.7.132.129:41881): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c991e46989b24e1db557b54e0a2501b3: Network error: Client connection negotiation failed: client connection to 127.7.132.129:41881: connect: Connection refused (error 111)
I20260812 02:40:30.938427 27672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:40:30.923281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60144 (local address 127.7.132.130:37905)
0812 02:40:30.923452 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0812 02:40:30.923456 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 02:40:30.923473 (+ 17us) server_negotiation.cc:407] Connection header received
0812 02:40:30.923528 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:40:30.923531 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:40:30.923590 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:40:30.923703 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:40:30.925679 (+ 1976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:40:30.926504 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:40:30.928459 (+ 1955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:40:30.928961 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:40:30.929057 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:40:30.932405 (+ 3348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:40:30.932419 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:40:30.932433 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:40:30.932465 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:40:30.935416 (+ 2951us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:40:30.936301 (+ 885us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:40:30.936308 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:40:30.936310 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:40:30.936436 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:40:30.936854 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:40:30.936857 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:40:30.936859 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:40:30.937251 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:40:30.937475 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0812 02:40:30.937826 (+ 351us) server_negotiation.cc:299] Negotiation successful
0812 02:40:30.937991 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 02:40:30.939256 27501 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:40:30.978317 27673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:40:30.977611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:41881 (local address 127.0.0.1:59464)
0812 02:40:30.977908 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0812 02:40:30.978051 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.129:41881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113}
I20260812 02:40:30.979611 27673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:40:30.979174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:41881 (local address 127.0.0.1:59466)
0812 02:40:30.979351 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0812 02:40:30.979462 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.129:41881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260812 02:40:31.030335 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 27355
I20260812 02:40:31.040659 27673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:40:31.040100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:41881 (local address 127.0.0.1:59468)
0812 02:40:31.040304 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0812 02:40:31.040454 (+ 150us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.129:41881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260812 02:40:31.042201 27673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:40:31.041685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:41881 (local address 127.0.0.1:59472)
0812 02:40:31.041898 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0812 02:40:31.042004 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.129:41881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260812 02:40:31.055202 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 27509
I20260812 02:40:31.070552 27674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:40:31.056031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41628 (local address 127.7.132.131:34721)
0812 02:40:31.056234 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0812 02:40:31.056240 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:40:31.056352 (+ 112us) server_negotiation.cc:407] Connection header received
0812 02:40:31.056588 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:40:31.056593 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:40:31.056660 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:40:31.056788 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:40:31.058221 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:40:31.059093 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:40:31.060489 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:40:31.060962 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:40:31.061111 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:40:31.063804 (+ 2693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:40:31.063816 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:40:31.063831 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:40:31.063872 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:40:31.067275 (+ 3403us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:40:31.068094 (+ 819us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:40:31.068102 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:40:31.068104 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:40:31.068267 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:40:31.068701 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:40:31.068708 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:40:31.068711 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:40:31.069106 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:40:31.069450 (+ 344us) server_negotiation.cc:1035] Waiting for connection context
0812 02:40:31.069782 (+ 332us) server_negotiation.cc:299] Negotiation successful
0812 02:40:31.070019 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260812 02:40:31.071362 27653 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:40:31.122995 27673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:40:31.122407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:41881 (local address 127.0.0.1:59486)
0812 02:40:31.122660 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0812 02:40:31.122778 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.129:41881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260812 02:40:31.124142 27673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:40:31.123704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:41881 (local address 127.0.0.1:59502)
0812 02:40:31.123920 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0812 02:40:31.124010 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.129:41881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109}
I20260812 02:40:31.195833 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 27509
I20260812 02:40:31.221208 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 27092
I20260812 02:40:31.222510 27169 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:40:31.337148 27673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:40:31.336388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:41881 (local address 127.0.0.1:59514)
0812 02:40:31.336728 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0812 02:40:31.336920 (+ 192us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.129:41881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162}
I20260812 02:40:31.337198 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 27092
I20260812 02:40:51.369626 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:42683
--webserver_interface=127.7.132.190
--webserver_port=41403
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:44005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.132.190:42683
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:40:51.725244 27710 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:40:51.725548 27710 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:40:51.725740 27710 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:40:51.732986 27710 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260812 02:40:51.733093 27710 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260812 02:40:51.733182 27710 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260812 02:40:51.733259 27710 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:40:51.733352 27710 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:40:51.733484 27710 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:40:51.733556 27710 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:40:51.733640 27710 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:40:51.743747 27710 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:44005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.7.132.190:42683
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:42683
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.132.190
--webserver_port=41403
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:40:51.745108 27710 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:40:51.746519 27710 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:40:51.758948 27716 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:40:51.758982 27715 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:40:51.758972 27718 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:40:51.759853 27710 server_base.cc:1034] running on GCE node
I20260812 02:40:51.760784 27710 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:40:51.762240 27710 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:40:51.763486 27710 hybrid_clock.cc:648] HybridClock initialized: now 1786502451763396 us; error 61 us; skew 500 ppm
I20260812 02:40:51.763815 27710 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:40:51 dist-test-slave-mn4r krb5kdc[26436](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502451, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:40:51.767167 27710 init.cc:377] Logged in from keytab as kudu/127.7.132.190@KRBTEST.COM (short username kudu)
I20260812 02:40:51.768639 27710 webserver.cc:460] Webserver started at http://127.7.132.190:41403/ using document root <none> and password file <none>
I20260812 02:40:51.769358 27710 fs_manager.cc:359] Metadata directory not provided
I20260812 02:40:51.769495 27710 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:40:51.775134 27710 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260812 02:40:51.778060 27727 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:40:51.779441 27710 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 02:40:51.779631 27710 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/master-0/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/master-0/wal
uuid: "0fad46d3a46c4597903b423dfd124d70"
format_stamp: "Formatted at 2026-08-12 02:40:05 on dist-test-slave-mn4r"
server_key: "b731db17ef27b79895faecff3c1d5c27"
server_key_iv: "ed35bab47187d559b4b58d26a970b636"
server_key_version: "encryptionkey@0"
I20260812 02:40:51.780704 27710 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:40:51.817392 27710 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:40:51.822192 27710 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:40:51.822511 27710 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:40:51.841653 27710 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.190:42683
I20260812 02:40:51.841701 27784 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.190:42683 every 8 connection(s)
I20260812 02:40:51.843287 27710 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
I20260812 02:40:51.850698 27788 sys_catalog.cc:263] Verifying existing consensus state
I20260812 02:40:51.852581 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 27710
I20260812 02:40:51.853442 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.129:41881
--local_ip_for_outbound_sockets=127.7.132.129
--tserver_master_addrs=127.7.132.190:42683
--webserver_port=33519
--webserver_interface=127.7.132.129
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.129.keytab
--principal=kudu/127.7.132.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:44005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 02:40:51.854562 27788 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0fad46d3a46c4597903b423dfd124d70: Bootstrap starting.
I20260812 02:40:51.881528 27788 log.cc:824] T 00000000000000000000000000000000 P 0fad46d3a46c4597903b423dfd124d70: Log is configured to *not* fsync() on all Append() calls
I20260812 02:40:51.906338 27788 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0fad46d3a46c4597903b423dfd124d70: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 02:40:51.907119 27788 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0fad46d3a46c4597903b423dfd124d70: Bootstrap complete.
I20260812 02:40:51.922017 27788 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0fad46d3a46c4597903b423dfd124d70 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fad46d3a46c4597903b423dfd124d70" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 42683 } }
I20260812 02:40:51.923501 27788 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0fad46d3a46c4597903b423dfd124d70 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fad46d3a46c4597903b423dfd124d70, State: Initialized, Role: FOLLOWER
I20260812 02:40:51.924340 27788 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0fad46d3a46c4597903b423dfd124d70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fad46d3a46c4597903b423dfd124d70" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 42683 } }
I20260812 02:40:51.924679 27788 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0fad46d3a46c4597903b423dfd124d70 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 02:40:51.924826 27788 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0fad46d3a46c4597903b423dfd124d70 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 02:40:51.925053 27788 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0fad46d3a46c4597903b423dfd124d70 [term 2 FOLLOWER]: Advancing to term 3
I20260812 02:40:51.931664 27788 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0fad46d3a46c4597903b423dfd124d70 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fad46d3a46c4597903b423dfd124d70" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 42683 } }
I20260812 02:40:51.932304 27788 leader_election.cc:302] T 00000000000000000000000000000000 P 0fad46d3a46c4597903b423dfd124d70 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0fad46d3a46c4597903b423dfd124d70; no voters:
I20260812 02:40:51.933059 27788 leader_election.cc:288] T 00000000000000000000000000000000 P 0fad46d3a46c4597903b423dfd124d70 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260812 02:40:51.933211 27793 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0fad46d3a46c4597903b423dfd124d70 [term 3 FOLLOWER]: Leader election won for term 3
I20260812 02:40:51.934459 27793 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0fad46d3a46c4597903b423dfd124d70 [term 3 LEADER]: Becoming Leader. State: Replica: 0fad46d3a46c4597903b423dfd124d70, State: Running, Role: LEADER
I20260812 02:40:51.935014 27793 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0fad46d3a46c4597903b423dfd124d70 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fad46d3a46c4597903b423dfd124d70" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 42683 } }
I20260812 02:40:51.935544 27788 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0fad46d3a46c4597903b423dfd124d70 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:40:51.937742 27796 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0fad46d3a46c4597903b423dfd124d70 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "0fad46d3a46c4597903b423dfd124d70" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fad46d3a46c4597903b423dfd124d70" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 42683 } } }
I20260812 02:40:51.937768 27797 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0fad46d3a46c4597903b423dfd124d70 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0fad46d3a46c4597903b423dfd124d70. Latest consensus state: current_term: 3 leader_uuid: "0fad46d3a46c4597903b423dfd124d70" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fad46d3a46c4597903b423dfd124d70" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 42683 } } }
I20260812 02:40:51.938222 27796 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0fad46d3a46c4597903b423dfd124d70 [sys.catalog]: This master's current role is: LEADER
I20260812 02:40:51.938225 27797 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0fad46d3a46c4597903b423dfd124d70 [sys.catalog]: This master's current role is: LEADER
I20260812 02:40:51.939138 27800 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:40:51.959054 27800 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 02:40:51.960632 27800 catalog_manager.cc:1259] Loaded cluster ID: 40c1dacf063b4f2d90266628e0db9540
I20260812 02:40:51.960810 27800 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 02:40:51.965173 27800 catalog_manager.cc:1504] Loading token signing keys...
I20260812 02:40:51.966900 27800 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 0fad46d3a46c4597903b423dfd124d70: Loaded TSK: 0
I20260812 02:40:51.967137 27800 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 0fad46d3a46c4597903b423dfd124d70: Loaded TSK: 1
F20260812 02:40:51.968319 27800 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f8f7982b9ad google::LogMessage::Fail() at ??:0
@ 0x7f8f7982e88a google::LogMessage::SendToLog() at ??:0
@ 0x7f8f7982b4de google::LogMessage::Flush() at ??:0
@ 0x7f8f7982ca19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f8f7d58f7d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f8f7d57e73e kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f8f8f25a1c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f8f8f259783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f8f8f2e52a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7f8f8f2e50c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7f8f8f32f5e9 std::function<>::operator()() at ??:0
@ 0x7f8f8f25dcdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f8f8f25bc36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f8f8f2e3610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f8f8f2e3439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f8f7c477345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f8f7c47a91c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f8f7c47a799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f8f7c455eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f8f7f72f6db start_thread at ??:0
@ 0x7f8f786c561f clone at ??:0
W20260812 02:40:52.317067 27791 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:40:52.317375 27791 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:40:52.317551 27791 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:40:52.325069 27791 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:40:52.325227 27791 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:40:52.325425 27791 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.129
I20260812 02:40:52.335870 27791 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:44005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.129.keytab
--principal=kudu/127.7.132.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.129:41881
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.132.129
--webserver_port=33519
--tserver_master_addrs=127.7.132.190:42683
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.129
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:40:52.337352 27791 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:40:52.338835 27791 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:40:52.353492 27845 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:40:52.353535 27846 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:40:52.354172 27848 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:40:52.354612 27791 server_base.cc:1034] running on GCE node
I20260812 02:40:52.355296 27791 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:40:52.356335 27791 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:40:52.357601 27791 hybrid_clock.cc:648] HybridClock initialized: now 1786502452357549 us; error 36 us; skew 500 ppm
I20260812 02:40:52.357966 27791 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:40:52 dist-test-slave-mn4r krb5kdc[26436](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502452, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:40:52.361210 27791 init.cc:377] Logged in from keytab as kudu/127.7.132.129@KRBTEST.COM (short username kudu)
I20260812 02:40:52.362798 27791 webserver.cc:460] Webserver started at http://127.7.132.129:33519/ using document root <none> and password file <none>
I20260812 02:40:52.363487 27791 fs_manager.cc:359] Metadata directory not provided
I20260812 02:40:52.363592 27791 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:40:52.369367 27791 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 02:40:52.372316 27856 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:40:52.373695 27791 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 02:40:52.373872 27791 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-0/wal
uuid: "cabf179ab4374509b0163a571e20f483"
format_stamp: "Formatted at 2026-08-12 02:40:06 on dist-test-slave-mn4r"
server_key: "dfe9a185d5e7020d0b76ae28c460f70b"
server_key_iv: "77e6c572a50b660125dad0e8fca42854"
server_key_version: "encryptionkey@0"
I20260812 02:40:52.374926 27791 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:40:52.390280 27791 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:40:52.394363 27791 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:40:52.394666 27791 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:40:52.395690 27791 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:40:52.397569 27864 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260812 02:40:52.408964 27791 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 02:40:52.409132 27791 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.012s user 0.001s sys 0.000s
I20260812 02:40:52.409281 27791 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260812 02:40:52.412550 27791 ts_tablet_manager.cc:610] Registered 1 tablets
I20260812 02:40:52.412662 27791 ts_tablet_manager.cc:589] Time spent register tablets: real 0.003s user 0.001s sys 0.000s
I20260812 02:40:52.412997 27864 tablet_bootstrap.cc:492] T c991e46989b24e1db557b54e0a2501b3 P cabf179ab4374509b0163a571e20f483: Bootstrap starting.
I20260812 02:40:52.442866 27864 log.cc:824] T c991e46989b24e1db557b54e0a2501b3 P cabf179ab4374509b0163a571e20f483: Log is configured to *not* fsync() on all Append() calls
I20260812 02:40:52.457543 27864 tablet_bootstrap.cc:492] T c991e46989b24e1db557b54e0a2501b3 P cabf179ab4374509b0163a571e20f483: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 02:40:52.458293 27864 tablet_bootstrap.cc:492] T c991e46989b24e1db557b54e0a2501b3 P cabf179ab4374509b0163a571e20f483: Bootstrap complete.
I20260812 02:40:52.460362 27864 ts_tablet_manager.cc:1397] T c991e46989b24e1db557b54e0a2501b3 P cabf179ab4374509b0163a571e20f483: Time spent bootstrapping tablet: real 0.048s user 0.020s sys 0.025s
I20260812 02:40:52.464613 27791 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.129:41881
I20260812 02:40:52.464705 28001 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.129:41881 every 8 connection(s)
I20260812 02:40:52.465969 27791 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-0/data/info.pb
I20260812 02:40:52.469029 27864 raft_consensus.cc:348] T c991e46989b24e1db557b54e0a2501b3 P cabf179ab4374509b0163a571e20f483 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cabf179ab4374509b0163a571e20f483" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 41881 } } peers { permanent_uuid: "c2298009914a4733a6b0604dd0843833" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37905 } } peers { permanent_uuid: "6ae48edd11834d13a65cb21c37afb69a" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 34721 } }
I20260812 02:40:52.470468 27864 raft_consensus.cc:729] T c991e46989b24e1db557b54e0a2501b3 P cabf179ab4374509b0163a571e20f483 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cabf179ab4374509b0163a571e20f483, State: Initialized, Role: FOLLOWER
I20260812 02:40:52.471200 27864 consensus_queue.cc:260] T c991e46989b24e1db557b54e0a2501b3 P cabf179ab4374509b0163a571e20f483 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cabf179ab4374509b0163a571e20f483" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 41881 } } peers { permanent_uuid: "c2298009914a4733a6b0604dd0843833" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37905 } } peers { permanent_uuid: "6ae48edd11834d13a65cb21c37afb69a" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 34721 } }
I20260812 02:40:52.471493 28004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:40:52.468919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:42683 (local address 127.7.132.129:57425)
0812 02:40:52.469876 (+ 957us) negotiation.cc:107] Waiting for socket to connect
0812 02:40:52.470232 (+ 356us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:42683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":598,"thread_start_us":379,"threads_started":1}
W20260812 02:40:52.472507 28002 heartbeater.cc:644] Failed to heartbeat to 127.7.132.190:42683 (0 consecutive failures): Network error: Failed to ping master at 127.7.132.190:42683: Client connection negotiation failed: client connection to 127.7.132.190:42683: connect: Connection refused (error 111)
I20260812 02:40:52.472692 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 27791
I20260812 02:40:52.473023 27864 ts_tablet_manager.cc:1428] T c991e46989b24e1db557b54e0a2501b3 P cabf179ab4374509b0163a571e20f483: Time spent starting tablet: real 0.013s user 0.004s sys 0.008s
I20260812 02:40:52.473526 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.130:37905
--local_ip_for_outbound_sockets=127.7.132.130
--tserver_master_addrs=127.7.132.190:42683
--webserver_port=36611
--webserver_interface=127.7.132.130
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.130.keytab
--principal=kudu/127.7.132.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:44005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 02:40:52.474422 28004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:40:52.473066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:42683 (local address 127.7.132.129:47445)
0812 02:40:52.474241 (+ 1175us) negotiation.cc:107] Waiting for socket to connect
0812 02:40:52.474312 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:42683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1079}
I20260812 02:40:52.475448 28004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:40:52.474980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:42683 (local address 127.7.132.129:44085)
0812 02:40:52.475169 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0812 02:40:52.475279 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:42683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260812 02:40:52.475728 28002 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 02:40:52.827497 28007 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:40:52.827817 28007 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:40:52.828001 28007 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:40:52.835570 28007 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:40:52.835695 28007 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:40:52.835883 28007 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.130
I20260812 02:40:52.846249 28007 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:44005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.130.keytab
--principal=kudu/127.7.132.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.130:37905
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.132.130
--webserver_port=36611
--tserver_master_addrs=127.7.132.190:42683
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.130
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:40:52.847643 28007 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:40:52.849084 28007 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:40:52.863261 28013 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:40:52.863339 28014 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:40:52.863310 28018 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:40:52.864261 28007 server_base.cc:1034] running on GCE node
I20260812 02:40:52.865015 28007 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:40:52.866166 28007 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:40:52.867467 28007 hybrid_clock.cc:648] HybridClock initialized: now 1786502452867392 us; error 51 us; skew 500 ppm
I20260812 02:40:52.867874 28007 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:40:52 dist-test-slave-mn4r krb5kdc[26436](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502452, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:40:52.871222 28007 init.cc:377] Logged in from keytab as kudu/127.7.132.130@KRBTEST.COM (short username kudu)
I20260812 02:40:52.872902 28007 webserver.cc:460] Webserver started at http://127.7.132.130:36611/ using document root <none> and password file <none>
I20260812 02:40:52.873610 28007 fs_manager.cc:359] Metadata directory not provided
I20260812 02:40:52.873718 28007 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:40:52.879623 28007 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 02:40:52.882819 28026 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:40:52.884724 28007 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260812 02:40:52.884977 28007 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-1/wal
uuid: "c2298009914a4733a6b0604dd0843833"
format_stamp: "Formatted at 2026-08-12 02:40:06 on dist-test-slave-mn4r"
server_key: "1b234fb35756c4ca2b9a5ad2ac41f3fd"
server_key_iv: "0df15e5f009641f6ebcb4ec45d2108ba"
server_key_version: "encryptionkey@0"
I20260812 02:40:52.886276 28007 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:40:52.910514 28007 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:40:52.915701 28007 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:40:52.916087 28007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:40:52.917207 28007 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:40:52.919075 28007 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:40:52.919227 28007 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:40:52.919378 28007 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:40:52.919469 28007 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:40:52.957464 28007 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.130:37905
I20260812 02:40:52.957557 28155 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.130:37905 every 8 connection(s)
I20260812 02:40:52.959139 28007 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-1/data/info.pb
I20260812 02:40:52.964442 28158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:40:52.962451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:42683 (local address 127.7.132.130:52381)
0812 02:40:52.963248 (+ 797us) negotiation.cc:107] Waiting for socket to connect
0812 02:40:52.963504 (+ 256us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:42683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":519,"thread_start_us":297,"threads_started":1}
W20260812 02:40:52.965154 28156 heartbeater.cc:644] Failed to heartbeat to 127.7.132.190:42683 (0 consecutive failures): Network error: Failed to ping master at 127.7.132.190:42683: Client connection negotiation failed: client connection to 127.7.132.190:42683: connect: Connection refused (error 111)
I20260812 02:40:52.965899 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 28007
I20260812 02:40:52.966126 28158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:40:52.965716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:42683 (local address 127.7.132.130:41761)
0812 02:40:52.965875 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0812 02:40:52.965954 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:42683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260812 02:40:52.966639 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.131:34721
--local_ip_for_outbound_sockets=127.7.132.131
--tserver_master_addrs=127.7.132.190:42683
--webserver_port=40209
--webserver_interface=127.7.132.131
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.131.keytab
--principal=kudu/127.7.132.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:44005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 02:40:52.967126 28158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:40:52.966771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:42683 (local address 127.7.132.130:59709)
0812 02:40:52.966922 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0812 02:40:52.967008 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:42683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260812 02:40:52.967374 28156 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 02:40:53.313763 28159 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:40:53.314177 28159 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:40:53.314390 28159 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:40:53.322314 28159 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:40:53.322482 28159 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:40:53.322690 28159 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.131
I20260812 02:40:53.333791 28159 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:44005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.131.keytab
--principal=kudu/127.7.132.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.131:34721
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.132.131
--webserver_port=40209
--tserver_master_addrs=127.7.132.190:42683
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.131
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:40:53.335316 28159 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:40:53.336889 28159 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:40:53.354063 28165 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:40:53.354174 28168 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:40:53.354012 28166 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:40:53.355141 28159 server_base.cc:1034] running on GCE node
I20260812 02:40:53.355793 28159 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:40:53.356830 28159 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:40:53.358120 28159 hybrid_clock.cc:648] HybridClock initialized: now 1786502453358010 us; error 66 us; skew 500 ppm
I20260812 02:40:53.358460 28159 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:40:53 dist-test-slave-mn4r krb5kdc[26436](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502453, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:40:53.361773 28159 init.cc:377] Logged in from keytab as kudu/127.7.132.131@KRBTEST.COM (short username kudu)
I20260812 02:40:53.363412 28159 webserver.cc:460] Webserver started at http://127.7.132.131:40209/ using document root <none> and password file <none>
I20260812 02:40:53.364204 28159 fs_manager.cc:359] Metadata directory not provided
I20260812 02:40:53.364353 28159 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:40:53.369928 28159 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260812 02:40:53.372853 28175 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:40:53.374414 28159 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 02:40:53.374598 28159 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-2/wal
uuid: "6ae48edd11834d13a65cb21c37afb69a"
format_stamp: "Formatted at 2026-08-12 02:40:07 on dist-test-slave-mn4r"
server_key: "ffd82d769373b97c9960a53afc1bfa1e"
server_key_iv: "1ae78bcf9dea599eaaa8593e58113d0c"
server_key_version: "encryptionkey@0"
I20260812 02:40:53.375630 28159 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:40:53.397621 28159 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:40:53.402274 28159 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:40:53.402563 28159 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:40:53.403614 28159 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:40:53.404989 28159 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:40:53.405123 28159 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:40:53.405264 28159 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:40:53.405355 28159 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:40:53.442831 28159 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.131:34721
I20260812 02:40:53.442924 28307 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.131:34721 every 8 connection(s)
I20260812 02:40:53.444330 28159 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786502156971023-7698-0/minicluster-data/ts-2/data/info.pb
I20260812 02:40:53.448292 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 28159
I20260812 02:40:53.450057 28311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:40:53.447484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:42683 (local address 127.7.132.131:47601)
0812 02:40:53.448475 (+ 991us) negotiation.cc:107] Waiting for socket to connect
0812 02:40:53.448781 (+ 306us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:42683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":590,"thread_start_us":334,"threads_started":1}
W20260812 02:40:53.450807 28308 heartbeater.cc:644] Failed to heartbeat to 127.7.132.190:42683 (0 consecutive failures): Network error: Failed to ping master at 127.7.132.190:42683: Client connection negotiation failed: client connection to 127.7.132.190:42683: connect: Connection refused (error 111)
I20260812 02:40:53.451685 28311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:40:53.451302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:42683 (local address 127.7.132.131:36689)
0812 02:40:53.451483 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0812 02:40:53.451557 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:42683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260812 02:40:53.452706 28311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:40:53.452358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:42683 (local address 127.7.132.131:60835)
0812 02:40:53.452510 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0812 02:40:53.452596 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:42683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260812 02:40:53.453012 28308 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 02:40:53.476930 28313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:40:53.476224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:42683 (local address 127.7.132.129:45563)
0812 02:40:53.476653 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0812 02:40:53.476740 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:42683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":206,"threads_started":1}
I20260812 02:40:53.932977 28314 raft_consensus.cc:482] T c991e46989b24e1db557b54e0a2501b3 P cabf179ab4374509b0163a571e20f483 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 02:40:53.933316 28314 raft_consensus.cc:504] T c991e46989b24e1db557b54e0a2501b3 P cabf179ab4374509b0163a571e20f483 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cabf179ab4374509b0163a571e20f483" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 41881 } } peers { permanent_uuid: "c2298009914a4733a6b0604dd0843833" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37905 } } peers { permanent_uuid: "6ae48edd11834d13a65cb21c37afb69a" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 34721 } }
I20260812 02:40:53.935173 28314 leader_election.cc:288] T c991e46989b24e1db557b54e0a2501b3 P cabf179ab4374509b0163a571e20f483 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c2298009914a4733a6b0604dd0843833 (127.7.132.130:37905), 6ae48edd11834d13a65cb21c37afb69a (127.7.132.131:34721)
Aug 12 02:40:53 dist-test-slave-mn4r krb5kdc[26436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502452, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.129@KRBTEST.COM for kudu/127.7.132.131@KRBTEST.COM
Aug 12 02:40:53 dist-test-slave-mn4r krb5kdc[26436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502452, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.129@KRBTEST.COM for kudu/127.7.132.130@KRBTEST.COM
I20260812 02:40:53.959372 28313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:40:53.935460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.131:34721 (local address 127.7.132.129:40923)
0812 02:40:53.935814 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0812 02:40:53.935862 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 02:40:53.936656 (+ 794us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:40:53.940900 (+ 4244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:40:53.940906 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:40:53.942133 (+ 1227us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:40:53.943277 (+ 1144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:40:53.943307 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:40:53.944935 (+ 1628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:40:53.944939 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:40:53.946391 (+ 1452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:40:53.946405 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:40:53.947412 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:40:53.947418 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:40:53.947691 (+ 273us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:40:53.948473 (+ 782us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:40:53.948496 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:40:53.951728 (+ 3232us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:40:53.955400 (+ 3672us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:40:53.955410 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:40:53.955426 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:40:53.955881 (+ 455us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:40:53.956393 (+ 512us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:40:53.956400 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:40:53.956406 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:40:53.956692 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:40:53.957722 (+ 1030us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:40:53.957730 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:40:53.958044 (+ 314us) client_negotiation.cc:769] Sending connection context
0812 02:40:53.958408 (+ 364us) client_negotiation.cc:240] Negotiation successful
0812 02:40:53.958931 (+ 523us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"mutex_wait_us":109,"spinlock_wait_cycles":31488}
I20260812 02:40:53.959550 28317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:40:53.935993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:40923 (local address 127.7.132.131:34721)
0812 02:40:53.938245 (+ 2252us) server_negotiation.cc:206] Beginning negotiation
0812 02:40:53.938261 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 02:40:53.938311 (+ 50us) server_negotiation.cc:407] Connection header received
0812 02:40:53.939920 (+ 1609us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:40:53.939930 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:40:53.940411 (+ 481us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:40:53.940755 (+ 344us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:40:53.943520 (+ 2765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:40:53.944762 (+ 1242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:40:53.946544 (+ 1782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:40:53.947214 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:40:53.947352 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:40:53.952040 (+ 4688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:40:53.952048 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:40:53.952062 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:40:53.952099 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:40:53.955242 (+ 3143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:40:53.956094 (+ 852us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:40:53.956101 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:40:53.956106 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:40:53.956265 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:40:53.956869 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:40:53.956877 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:40:53.956882 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:40:53.957577 (+ 695us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:40:53.957781 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0812 02:40:53.958678 (+ 897us) server_negotiation.cc:299] Negotiation successful
0812 02:40:53.959125 (+ 447us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2014,"thread_start_us":1498,"threads_started":1}
I20260812 02:40:53.959757 28316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:40:53.935912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:38133 (local address 127.7.132.130:37905)
0812 02:40:53.938220 (+ 2308us) server_negotiation.cc:206] Beginning negotiation
0812 02:40:53.938233 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 02:40:53.938271 (+ 38us) server_negotiation.cc:407] Connection header received
0812 02:40:53.939922 (+ 1651us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:40:53.939933 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:40:53.940408 (+ 475us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:40:53.940763 (+ 355us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:40:53.944978 (+ 4215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:40:53.946387 (+ 1409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:40:53.947805 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:40:53.948459 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:40:53.948597 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:40:53.952640 (+ 4043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:40:53.952653 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:40:53.952671 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:40:53.952725 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:40:53.955869 (+ 3144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:40:53.956585 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:40:53.956592 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:40:53.956597 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:40:53.956821 (+ 224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:40:53.957302 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:40:53.957308 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:40:53.957311 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:40:53.958020 (+ 709us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:40:53.958262 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0812 02:40:53.958832 (+ 570us) server_negotiation.cc:299] Negotiation successful
0812 02:40:53.959247 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2113,"thread_start_us":1982,"threads_started":1}
I20260812 02:40:53.960644 28315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:40:53.935531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.130:37905 (local address 127.7.132.129:38133)
0812 02:40:53.937578 (+ 2047us) negotiation.cc:107] Waiting for socket to connect
0812 02:40:53.937601 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0812 02:40:53.937721 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:40:53.940888 (+ 3167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:40:53.940901 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:40:53.942133 (+ 1232us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:40:53.943277 (+ 1144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:40:53.943307 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:40:53.946571 (+ 3264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:40:53.946583 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:40:53.947656 (+ 1073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:40:53.947669 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:40:53.948664 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:40:53.948673 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:40:53.948895 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:40:53.950100 (+ 1205us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:40:53.950134 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:40:53.952321 (+ 2187us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:40:53.956022 (+ 3701us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:40:53.956030 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:40:53.956036 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:40:53.956396 (+ 360us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:40:53.956978 (+ 582us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:40:53.956983 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:40:53.956986 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:40:53.957176 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:40:53.958215 (+ 1039us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:40:53.958220 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:40:53.958364 (+ 144us) client_negotiation.cc:769] Sending connection context
0812 02:40:53.958526 (+ 162us) client_negotiation.cc:240] Negotiation successful
0812 02:40:53.960336 (+ 1810us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1900,"mutex_wait_us":1685,"spinlock_wait_cycles":18816,"thread_start_us":1690,"threads_started":1}
W20260812 02:40:53.964346 27860 leader_election.cc:341] T c991e46989b24e1db557b54e0a2501b3 P cabf179ab4374509b0163a571e20f483 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c2298009914a4733a6b0604dd0843833 (127.7.132.130:37905): Not found: Tablet not found: c991e46989b24e1db557b54e0a2501b3
W20260812 02:40:53.965472 27857 leader_election.cc:341] T c991e46989b24e1db557b54e0a2501b3 P cabf179ab4374509b0163a571e20f483 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6ae48edd11834d13a65cb21c37afb69a (127.7.132.131:34721): Not found: Tablet not found: c991e46989b24e1db557b54e0a2501b3
I20260812 02:40:53.965636 27857 leader_election.cc:302] T c991e46989b24e1db557b54e0a2501b3 P cabf179ab4374509b0163a571e20f483 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cabf179ab4374509b0163a571e20f483; no voters: 6ae48edd11834d13a65cb21c37afb69a, c2298009914a4733a6b0604dd0843833
I20260812 02:40:53.966153 28314 raft_consensus.cc:2726] T c991e46989b24e1db557b54e0a2501b3 P cabf179ab4374509b0163a571e20f483 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 02:40:53.968662 28318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:40:53.967815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:42683 (local address 127.7.132.130:46231)
0812 02:40:53.968269 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0812 02:40:53.968448 (+ 179us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:42683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":226,"threads_started":1}
I20260812 02:40:54.454545 28320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:40:54.453572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:42683 (local address 127.7.132.131:53719)
0812 02:40:54.454166 (+ 594us) negotiation.cc:107] Waiting for socket to connect
0812 02:40:54.454304 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:42683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":284,"threads_started":1}
I20260812 02:40:54.478593 28321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:40:54.477851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:42683 (local address 127.7.132.129:43571)
0812 02:40:54.478355 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0812 02:40:54.478433 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:42683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":311,"threads_started":1}
I20260812 02:40:54.970278 28322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:40:54.969534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:42683 (local address 127.7.132.130:42091)
0812 02:40:54.970012 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0812 02:40:54.970118 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:42683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":254,"threads_started":1}
I20260812 02:40:55.456146 28323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:40:55.455426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:42683 (local address 127.7.132.131:42571)
0812 02:40:55.455871 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0812 02:40:55.455953 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:42683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":222,"threads_started":1}
I20260812 02:40:55.480465 28324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:40:55.479520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:42683 (local address 127.7.132.129:59701)
0812 02:40:55.480170 (+ 650us) negotiation.cc:107] Waiting for socket to connect
0812 02:40:55.480277 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:42683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":538,"thread_start_us":322,"threads_started":1}
I20260812 02:40:55.972074 28325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:40:55.971286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:42683 (local address 127.7.132.130:60067)
0812 02:40:55.971787 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0812 02:40:55.971869 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:42683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":276,"threads_started":1}
I20260812 02:40:56.159759 28326 raft_consensus.cc:482] T c991e46989b24e1db557b54e0a2501b3 P cabf179ab4374509b0163a571e20f483 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 02:40:56.159950 28326 raft_consensus.cc:504] T c991e46989b24e1db557b54e0a2501b3 P cabf179ab4374509b0163a571e20f483 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cabf179ab4374509b0163a571e20f483" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 41881 } } peers { permanent_uuid: "c2298009914a4733a6b0604dd0843833" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37905 } } peers { permanent_uuid: "6ae48edd11834d13a65cb21c37afb69a" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 34721 } }
I20260812 02:40:56.160641 28326 leader_election.cc:288] T c991e46989b24e1db557b54e0a2501b3 P cabf179ab4374509b0163a571e20f483 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c2298009914a4733a6b0604dd0843833 (127.7.132.130:37905), 6ae48edd11834d13a65cb21c37afb69a (127.7.132.131:34721)
W20260812 02:40:56.161688 27860 leader_election.cc:341] T c991e46989b24e1db557b54e0a2501b3 P cabf179ab4374509b0163a571e20f483 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c2298009914a4733a6b0604dd0843833 (127.7.132.130:37905): Not found: Tablet not found: c991e46989b24e1db557b54e0a2501b3
W20260812 02:40:56.161918 27857 leader_election.cc:341] T c991e46989b24e1db557b54e0a2501b3 P cabf179ab4374509b0163a571e20f483 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6ae48edd11834d13a65cb21c37afb69a (127.7.132.131:34721): Not found: Tablet not found: c991e46989b24e1db557b54e0a2501b3
I20260812 02:40:56.162041 27857 leader_election.cc:302] T c991e46989b24e1db557b54e0a2501b3 P cabf179ab4374509b0163a571e20f483 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cabf179ab4374509b0163a571e20f483; no voters: 6ae48edd11834d13a65cb21c37afb69a, c2298009914a4733a6b0604dd0843833
I20260812 02:40:56.162410 28326 raft_consensus.cc:2726] T c991e46989b24e1db557b54e0a2501b3 P cabf179ab4374509b0163a571e20f483 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 02:40:56.457738 28328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:40:56.457057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:42683 (local address 127.7.132.131:42669)
0812 02:40:56.457481 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0812 02:40:56.457571 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:42683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":227,"threads_started":1}
I20260812 02:40:56.482178 28329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:40:56.481320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:42683 (local address 127.7.132.129:44579)
0812 02:40:56.481783 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0812 02:40:56.481906 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:42683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":235,"threads_started":1}
I20260812 02:40:56.974187 28330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:40:56.973214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:42683 (local address 127.7.132.130:43339)
0812 02:40:56.973815 (+ 601us) negotiation.cc:107] Waiting for socket to connect
0812 02:40:56.973942 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:42683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":302,"threads_started":1}
I20260812 02:40:57.459314 28331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:40:57.458598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:42683 (local address 127.7.132.131:35093)
0812 02:40:57.459047 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0812 02:40:57.459126 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:42683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":223,"threads_started":1}
I20260812 02:40:57.483994 28332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:40:57.483228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:42683 (local address 127.7.132.129:40749)
0812 02:40:57.483712 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0812 02:40:57.483796 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:42683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":237,"threads_started":1}
I20260812 02:40:57.976014 28333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:40:57.975197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:42683 (local address 127.7.132.130:53421)
0812 02:40:57.975710 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0812 02:40:57.975808 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:42683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":265,"threads_started":1}
I20260812 02:40:58.088822 28334 raft_consensus.cc:482] T c991e46989b24e1db557b54e0a2501b3 P cabf179ab4374509b0163a571e20f483 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 02:40:58.089066 28334 raft_consensus.cc:504] T c991e46989b24e1db557b54e0a2501b3 P cabf179ab4374509b0163a571e20f483 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cabf179ab4374509b0163a571e20f483" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 41881 } } peers { permanent_uuid: "c2298009914a4733a6b0604dd0843833" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37905 } } peers { permanent_uuid: "6ae48edd11834d13a65cb21c37afb69a" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 34721 } }
I20260812 02:40:58.089803 28334 leader_election.cc:288] T c991e46989b24e1db557b54e0a2501b3 P cabf179ab4374509b0163a571e20f483 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c2298009914a4733a6b0604dd0843833 (127.7.132.130:37905), 6ae48edd11834d13a65cb21c37afb69a (127.7.132.131:34721)
W20260812 02:40:58.090891 27860 leader_election.cc:341] T c991e46989b24e1db557b54e0a2501b3 P cabf179ab4374509b0163a571e20f483 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c2298009914a4733a6b0604dd0843833 (127.7.132.130:37905): Not found: Tablet not found: c991e46989b24e1db557b54e0a2501b3
W20260812 02:40:58.091091 27857 leader_election.cc:341] T c991e46989b24e1db557b54e0a2501b3 P cabf179ab4374509b0163a571e20f483 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6ae48edd11834d13a65cb21c37afb69a (127.7.132.131:34721): Not found: Tablet not found: c991e46989b24e1db557b54e0a2501b3
I20260812 02:40:58.091249 27857 leader_election.cc:302] T c991e46989b24e1db557b54e0a2501b3 P cabf179ab4374509b0163a571e20f483 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cabf179ab4374509b0163a571e20f483; no voters: 6ae48edd11834d13a65cb21c37afb69a, c2298009914a4733a6b0604dd0843833
I20260812 02:40:58.091621 28334 raft_consensus.cc:2726] T c991e46989b24e1db557b54e0a2501b3 P cabf179ab4374509b0163a571e20f483 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 02:40:58.461069 28335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:40:58.460305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:42683 (local address 127.7.132.131:40637)
0812 02:40:58.460787 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0812 02:40:58.460884 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:42683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":251,"threads_started":1}
I20260812 02:40:58.485918 28336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:40:58.484937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:42683 (local address 127.7.132.129:41889)
0812 02:40:58.485591 (+ 654us) negotiation.cc:107] Waiting for socket to connect
0812 02:40:58.485716 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:42683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":526,"thread_start_us":324,"threads_started":1}
I20260812 02:40:58.977821 28337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:40:58.976996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:42683 (local address 127.7.132.130:47025)
0812 02:40:58.977519 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0812 02:40:58.977602 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:42683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":261,"threads_started":1}
I20260812 02:40:59.463126 28338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:40:59.462029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:42683 (local address 127.7.132.131:41627)
0812 02:40:59.462685 (+ 656us) negotiation.cc:107] Waiting for socket to connect
0812 02:40:59.462827 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:42683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":488,"thread_start_us":318,"threads_started":1}
I20260812 02:40:59.487778 28339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:40:59.487014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:42683 (local address 127.7.132.129:37295)
0812 02:40:59.487494 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0812 02:40:59.487579 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:42683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":249,"threads_started":1}
I20260812 02:40:59.979749 28340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:40:59.978836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:42683 (local address 127.7.132.130:55941)
0812 02:40:59.979457 (+ 621us) negotiation.cc:107] Waiting for socket to connect
0812 02:40:59.979550 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:42683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":520,"thread_start_us":265,"threads_started":1}
I20260812 02:41:00.465234 28341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:00.464268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:42683 (local address 127.7.132.131:60553)
0812 02:41:00.464949 (+ 681us) negotiation.cc:107] Waiting for socket to connect
0812 02:41:00.465041 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:42683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":569,"thread_start_us":306,"threads_started":1}
I20260812 02:41:00.489487 28343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:00.488756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:42683 (local address 127.7.132.129:39361)
0812 02:41:00.489226 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0812 02:41:00.489318 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:42683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":262,"threads_started":1}
I20260812 02:41:00.981658 28344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:00.980827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:42683 (local address 127.7.132.130:51323)
0812 02:41:00.981374 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0812 02:41:00.981459 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:42683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":318,"threads_started":1}
I20260812 02:41:01.468415 28345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:01.466349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:42683 (local address 127.7.132.131:52483)
0812 02:41:01.468034 (+ 1685us) negotiation.cc:107] Waiting for socket to connect
0812 02:41:01.468158 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:42683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1552,"thread_start_us":1411,"threads_started":1}
I20260812 02:41:01.491272 28347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:01.490541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:42683 (local address 127.7.132.129:54341)
0812 02:41:01.490994 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0812 02:41:01.491096 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:42683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":234,"threads_started":1}
I20260812 02:41:01.983611 28349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:01.982761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:42683 (local address 127.7.132.130:47935)
0812 02:41:01.983362 (+ 601us) negotiation.cc:107] Waiting for socket to connect
0812 02:41:01.983452 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:42683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":349,"threads_started":1}
I20260812 02:41:02.352187 28350 raft_consensus.cc:482] T c991e46989b24e1db557b54e0a2501b3 P cabf179ab4374509b0163a571e20f483 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 02:41:02.352475 28350 raft_consensus.cc:504] T c991e46989b24e1db557b54e0a2501b3 P cabf179ab4374509b0163a571e20f483 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cabf179ab4374509b0163a571e20f483" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 41881 } } peers { permanent_uuid: "c2298009914a4733a6b0604dd0843833" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37905 } } peers { permanent_uuid: "6ae48edd11834d13a65cb21c37afb69a" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 34721 } }
I20260812 02:41:02.353245 28350 leader_election.cc:288] T c991e46989b24e1db557b54e0a2501b3 P cabf179ab4374509b0163a571e20f483 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c2298009914a4733a6b0604dd0843833 (127.7.132.130:37905), 6ae48edd11834d13a65cb21c37afb69a (127.7.132.131:34721)
W20260812 02:41:02.354228 27860 leader_election.cc:341] T c991e46989b24e1db557b54e0a2501b3 P cabf179ab4374509b0163a571e20f483 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c2298009914a4733a6b0604dd0843833 (127.7.132.130:37905): Not found: Tablet not found: c991e46989b24e1db557b54e0a2501b3
W20260812 02:41:02.354430 27857 leader_election.cc:341] T c991e46989b24e1db557b54e0a2501b3 P cabf179ab4374509b0163a571e20f483 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6ae48edd11834d13a65cb21c37afb69a (127.7.132.131:34721): Not found: Tablet not found: c991e46989b24e1db557b54e0a2501b3
I20260812 02:41:02.354574 27857 leader_election.cc:302] T c991e46989b24e1db557b54e0a2501b3 P cabf179ab4374509b0163a571e20f483 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cabf179ab4374509b0163a571e20f483; no voters: 6ae48edd11834d13a65cb21c37afb69a, c2298009914a4733a6b0604dd0843833
I20260812 02:41:02.354854 28350 raft_consensus.cc:2726] T c991e46989b24e1db557b54e0a2501b3 P cabf179ab4374509b0163a571e20f483 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 02:41:02.470850 28351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:02.469822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:42683 (local address 127.7.132.131:49791)
0812 02:41:02.470495 (+ 673us) negotiation.cc:107] Waiting for socket to connect
0812 02:41:02.470608 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:42683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":511,"thread_start_us":343,"threads_started":1}
I20260812 02:41:02.493441 28352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:02.492440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:42683 (local address 127.7.132.129:52607)
0812 02:41:02.493092 (+ 652us) negotiation.cc:107] Waiting for socket to connect
0812 02:41:02.493223 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:42683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":527,"thread_start_us":372,"threads_started":1}
I20260812 02:41:02.985404 28353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:02.984654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:42683 (local address 127.7.132.130:48973)
0812 02:41:02.985122 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0812 02:41:02.985209 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:42683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":243,"threads_started":1}
I20260812 02:41:03.472677 28354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:03.471894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:42683 (local address 127.7.132.131:49967)
0812 02:41:03.472367 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0812 02:41:03.472459 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:42683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":257,"threads_started":1}
I20260812 02:41:03.495280 28355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:03.494491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:42683 (local address 127.7.132.129:54267)
0812 02:41:03.494943 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0812 02:41:03.495094 (+ 151us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:42683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":230,"threads_started":1}
I20260812 02:41:03.987644 28356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:03.986600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:42683 (local address 127.7.132.130:33233)
0812 02:41:03.987319 (+ 719us) negotiation.cc:107] Waiting for socket to connect
0812 02:41:03.987414 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:42683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":587,"thread_start_us":411,"threads_started":1}
I20260812 02:41:04.474555 28357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:04.473725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:42683 (local address 127.7.132.131:53277)
0812 02:41:04.474294 (+ 569us) negotiation.cc:107] Waiting for socket to connect
0812 02:41:04.474386 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:42683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":247,"threads_started":1}
I20260812 02:41:04.497056 28359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:04.496277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:42683 (local address 127.7.132.129:51369)
0812 02:41:04.496789 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0812 02:41:04.496879 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:42683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":281,"threads_started":1}
I20260812 02:41:04.989327 28360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:04.988610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:42683 (local address 127.7.132.130:54581)
0812 02:41:04.989061 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0812 02:41:04.989144 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:42683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":244,"threads_started":1}
I20260812 02:41:05.476188 28361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:05.475474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:42683 (local address 127.7.132.131:34109)
0812 02:41:05.475939 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0812 02:41:05.476022 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:42683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":243,"threads_started":1}
I20260812 02:41:05.498858 28362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:05.498030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:42683 (local address 127.7.132.129:50629)
0812 02:41:05.498576 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0812 02:41:05.498669 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:42683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":321,"threads_started":1}
I20260812 02:41:05.991202 28363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:05.990373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:42683 (local address 127.7.132.130:43875)
0812 02:41:05.990893 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0812 02:41:05.991021 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:42683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":297,"threads_started":1}
I20260812 02:41:06.477918 28364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:06.477163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:42683 (local address 127.7.132.131:34389)
0812 02:41:06.477651 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0812 02:41:06.477745 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:42683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":240,"threads_started":1}
I20260812 02:41:06.500799 28365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:06.499937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:42683 (local address 127.7.132.129:52163)
0812 02:41:06.500497 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0812 02:41:06.500595 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:42683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":285,"threads_started":1}
I20260812 02:41:06.993033 28366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:06.992127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:42683 (local address 127.7.132.130:54305)
0812 02:41:06.992667 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0812 02:41:06.992774 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:42683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":269,"threads_started":1}
I20260812 02:41:07.480000 28367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:07.479008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:42683 (local address 127.7.132.131:51365)
0812 02:41:07.479689 (+ 681us) negotiation.cc:107] Waiting for socket to connect
0812 02:41:07.479798 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:42683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":514,"thread_start_us":359,"threads_started":1}
I20260812 02:41:07.503165 28368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:07.502301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:42683 (local address 127.7.132.129:42313)
0812 02:41:07.502807 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0812 02:41:07.502918 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:42683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":271,"threads_started":1}
I20260812 02:41:07.778821 28369 raft_consensus.cc:482] T c991e46989b24e1db557b54e0a2501b3 P cabf179ab4374509b0163a571e20f483 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 02:41:07.779088 28369 raft_consensus.cc:504] T c991e46989b24e1db557b54e0a2501b3 P cabf179ab4374509b0163a571e20f483 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cabf179ab4374509b0163a571e20f483" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 41881 } } peers { permanent_uuid: "c2298009914a4733a6b0604dd0843833" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37905 } } peers { permanent_uuid: "6ae48edd11834d13a65cb21c37afb69a" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 34721 } }
I20260812 02:41:07.780006 28369 leader_election.cc:288] T c991e46989b24e1db557b54e0a2501b3 P cabf179ab4374509b0163a571e20f483 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c2298009914a4733a6b0604dd0843833 (127.7.132.130:37905), 6ae48edd11834d13a65cb21c37afb69a (127.7.132.131:34721)
W20260812 02:41:07.781119 27860 leader_election.cc:341] T c991e46989b24e1db557b54e0a2501b3 P cabf179ab4374509b0163a571e20f483 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c2298009914a4733a6b0604dd0843833 (127.7.132.130:37905): Not found: Tablet not found: c991e46989b24e1db557b54e0a2501b3
W20260812 02:41:07.781344 27857 leader_election.cc:341] T c991e46989b24e1db557b54e0a2501b3 P cabf179ab4374509b0163a571e20f483 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6ae48edd11834d13a65cb21c37afb69a (127.7.132.131:34721): Not found: Tablet not found: c991e46989b24e1db557b54e0a2501b3
I20260812 02:41:07.781533 27857 leader_election.cc:302] T c991e46989b24e1db557b54e0a2501b3 P cabf179ab4374509b0163a571e20f483 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cabf179ab4374509b0163a571e20f483; no voters: 6ae48edd11834d13a65cb21c37afb69a, c2298009914a4733a6b0604dd0843833
I20260812 02:41:07.781787 28369 raft_consensus.cc:2726] T c991e46989b24e1db557b54e0a2501b3 P cabf179ab4374509b0163a571e20f483 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 02:41:07.995056 28370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:07.994165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:42683 (local address 127.7.132.130:54245)
0812 02:41:07.994642 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0812 02:41:07.994776 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:42683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":243,"threads_started":1}
I20260812 02:41:08.455565 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 27791
I20260812 02:41:08.477178 28371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:08.456962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51116 (local address 127.7.132.129:41881)
0812 02:41:08.457528 (+ 566us) server_negotiation.cc:206] Beginning negotiation
0812 02:41:08.457539 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 02:41:08.457578 (+ 39us) server_negotiation.cc:407] Connection header received
0812 02:41:08.457990 (+ 412us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:41:08.458000 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:41:08.458357 (+ 357us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:41:08.458505 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:41:08.460012 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:08.461135 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:08.462624 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:08.463231 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:08.463343 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:08.470879 (+ 7536us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:41:08.470894 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:41:08.470911 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:41:08.470952 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:41:08.473983 (+ 3031us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:41:08.474789 (+ 806us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:41:08.474797 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:41:08.474799 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:41:08.474952 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:41:08.475384 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:41:08.475391 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:41:08.475394 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:41:08.475805 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:41:08.476154 (+ 349us) server_negotiation.cc:1035] Waiting for connection context
0812 02:41:08.476509 (+ 355us) server_negotiation.cc:299] Negotiation successful
0812 02:41:08.476825 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367,"thread_start_us":438,"threads_started":1}
I20260812 02:41:08.478538 27996 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:41:08.483908 28373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:08.481279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:42683 (local address 127.7.132.131:33047)
0812 02:41:08.483598 (+ 2319us) negotiation.cc:107] Waiting for socket to connect
0812 02:41:08.483737 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:42683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2200,"thread_start_us":2080,"threads_started":1}
I20260812 02:41:08.571676 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 27791
I20260812 02:41:08.595775 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 28007
I20260812 02:41:08.609406 28376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:08.596647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51694 (local address 127.7.132.130:37905)
0812 02:41:08.597291 (+ 644us) server_negotiation.cc:206] Beginning negotiation
0812 02:41:08.597296 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:41:08.597312 (+ 16us) server_negotiation.cc:407] Connection header received
0812 02:41:08.597370 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:41:08.597374 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:41:08.597434 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:41:08.597527 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:41:08.599041 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:08.599826 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:08.601151 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:08.601574 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:08.601666 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:08.604437 (+ 2771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:41:08.604442 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:41:08.604446 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:41:08.604477 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:41:08.607072 (+ 2595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:41:08.607674 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:41:08.607679 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:41:08.607681 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:41:08.607822 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:41:08.608145 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:41:08.608148 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:41:08.608150 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:41:08.608415 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:41:08.608613 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0812 02:41:08.608939 (+ 326us) server_negotiation.cc:299] Negotiation successful
0812 02:41:08.609111 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":548,"thread_start_us":406,"threads_started":1}
I20260812 02:41:08.610234 28151 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:41:08.694222 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 28007
I20260812 02:41:08.717873 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 28159
I20260812 02:41:08.731799 28378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:08.718736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57962 (local address 127.7.132.131:34721)
0812 02:41:08.719333 (+ 597us) server_negotiation.cc:206] Beginning negotiation
0812 02:41:08.719338 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:41:08.719356 (+ 18us) server_negotiation.cc:407] Connection header received
0812 02:41:08.719414 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:41:08.719418 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:41:08.719494 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:41:08.719588 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:41:08.721032 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:08.721912 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:08.723282 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:08.723751 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:08.723863 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:08.726624 (+ 2761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:41:08.726630 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:41:08.726633 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:41:08.726665 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:41:08.729352 (+ 2687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:41:08.729975 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:41:08.729979 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:41:08.729981 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:41:08.730199 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:41:08.730548 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:41:08.730552 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:41:08.730554 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:41:08.730809 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:41:08.731046 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0812 02:41:08.731317 (+ 271us) server_negotiation.cc:299] Negotiation successful
0812 02:41:08.731497 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":480,"thread_start_us":327,"threads_started":1}
I20260812 02:41:08.732544 28301 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:41:08.823576 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 28159
2026-08-12T02:41:08Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (65237 ms)
[----------] 30 tests from SecurityITest (311820 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502156971023-7698-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 02:41:08 dist-test-slave-mn4r krb5kdc[28385](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 02:41:08 dist-test-slave-mn4r krb5kdc[28385](info): set up 2 sockets
Aug 12 02:41:08 dist-test-slave-mn4r krb5kdc[28385](info): commencing operation
krb5kdc: starting...
W20260812 02:41:10.090925 7698 mini_kdc.cc:121] Time spent starting KDC: real 1.221s user 0.005s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502156971023-7698-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502156971023-7698-0/krb5kdc/test-user.keytab.
Aug 12 02:41:10 dist-test-slave-mn4r krb5kdc[28385](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502470, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T02:41:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T02:41:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.132.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
I20260812 02:41:10.403635 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502156971023-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502156971023-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502156971023-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:43367
--webserver_interface=127.7.132.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:45601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.132.190:43367
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:41:10.761782 28400 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:41:10.762140 28400 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:41:10.762323 28400 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:41:10.769763 28400 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:41:10.769919 28400 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:41:10.769997 28400 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:41:10.770056 28400 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:41:10.770155 28400 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:41:10.780261 28400 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:45601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502156971023-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502156971023-7698-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.132.190:43367
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:43367
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.132.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502156971023-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:41:10.781584 28400 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:41:10.783017 28400 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:41:10.795727 28408 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:41:10.795768 28412 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:41:10.796016 28400 server_base.cc:1034] running on GCE node
W20260812 02:41:10.796226 28407 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:41:10.797209 28400 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:41:10.798786 28400 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:41:10.800032 28400 hybrid_clock.cc:648] HybridClock initialized: now 1786502470799958 us; error 53 us; skew 500 ppm
I20260812 02:41:10.800390 28400 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:41:10 dist-test-slave-mn4r krb5kdc[28385](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502470, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:41:10.803725 28400 init.cc:377] Logged in from keytab as kudu/127.7.132.190@KRBTEST.COM (short username kudu)
I20260812 02:41:10.805187 28400 webserver.cc:460] Webserver started at http://127.7.132.190:38665/ using document root <none> and password file <none>
I20260812 02:41:10.805833 28400 fs_manager.cc:359] Metadata directory not provided
I20260812 02:41:10.805938 28400 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:41:10.806290 28400 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:41:10.809243 28400 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502156971023-7698-0/minicluster-data/master-0/data/instance:
uuid: "9556e0ddfe724780a25219b08a2083b7"
format_stamp: "Formatted at 2026-08-12 02:41:10 on dist-test-slave-mn4r"
server_key: "e2d70642bcb4c444e1814ca5fde513e3"
server_key_iv: "70d5d2b49104676faa430501c3813378"
server_key_version: "encryptionkey@0"
I20260812 02:41:10.809938 28400 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502156971023-7698-0/minicluster-data/master-0/wal/instance:
uuid: "9556e0ddfe724780a25219b08a2083b7"
format_stamp: "Formatted at 2026-08-12 02:41:10 on dist-test-slave-mn4r"
server_key: "e2d70642bcb4c444e1814ca5fde513e3"
server_key_iv: "70d5d2b49104676faa430501c3813378"
server_key_version: "encryptionkey@0"
I20260812 02:41:10.815215 28400 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260812 02:41:10.818580 28420 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:41:10.819922 28400 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260812 02:41:10.820104 28400 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502156971023-7698-0/minicluster-data/master-0/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502156971023-7698-0/minicluster-data/master-0/wal
uuid: "9556e0ddfe724780a25219b08a2083b7"
format_stamp: "Formatted at 2026-08-12 02:41:10 on dist-test-slave-mn4r"
server_key: "e2d70642bcb4c444e1814ca5fde513e3"
server_key_iv: "70d5d2b49104676faa430501c3813378"
server_key_version: "encryptionkey@0"
I20260812 02:41:10.820302 28400 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502156971023-7698-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502156971023-7698-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502156971023-7698-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:41:10.838150 28400 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:41:10.842329 28400 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:41:10.842675 28400 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:41:10.861953 28400 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.190:43367
I20260812 02:41:10.861984 28487 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.190:43367 every 8 connection(s)
I20260812 02:41:10.863394 28400 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
I20260812 02:41:10.867257 28488 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:41:10.873375 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 28400
I20260812 02:41:10.873639 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502156971023-7698-0/minicluster-data/master-0/wal/instance
I20260812 02:41:10.874555 7698 external_mini_cluster.cc:1428] Setting key c8fd2c68969eee6ecbab668fd7cf39c9
I20260812 02:41:10.878926 28488 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9556e0ddfe724780a25219b08a2083b7: Bootstrap starting.
I20260812 02:41:10.882843 28488 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9556e0ddfe724780a25219b08a2083b7: Neither blocks nor log segments found. Creating new log.
I20260812 02:41:10.884021 28488 log.cc:824] T 00000000000000000000000000000000 P 9556e0ddfe724780a25219b08a2083b7: Log is configured to *not* fsync() on all Append() calls
I20260812 02:41:10.887315 28488 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9556e0ddfe724780a25219b08a2083b7: No bootstrap required, opened a new log
Aug 12 02:41:10 dist-test-slave-mn4r krb5kdc[28385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502470, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 02:41:10.895879 28488 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9556e0ddfe724780a25219b08a2083b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9556e0ddfe724780a25219b08a2083b7" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 43367 } }
I20260812 02:41:10.896310 28488 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9556e0ddfe724780a25219b08a2083b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:41:10.896432 28488 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9556e0ddfe724780a25219b08a2083b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9556e0ddfe724780a25219b08a2083b7, State: Initialized, Role: FOLLOWER
I20260812 02:41:10.897065 28488 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9556e0ddfe724780a25219b08a2083b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9556e0ddfe724780a25219b08a2083b7" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 43367 } }
I20260812 02:41:10.897311 28488 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9556e0ddfe724780a25219b08a2083b7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 02:41:10.897466 28488 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9556e0ddfe724780a25219b08a2083b7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 02:41:10.897684 28488 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9556e0ddfe724780a25219b08a2083b7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:41:10.899304 28492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:10.876775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43036 (local address 127.7.132.190:43367)
0812 02:41:10.877576 (+ 801us) server_negotiation.cc:206] Beginning negotiation
0812 02:41:10.877588 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 02:41:10.877630 (+ 42us) server_negotiation.cc:407] Connection header received
0812 02:41:10.879464 (+ 1834us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:41:10.879486 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:41:10.879864 (+ 378us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:41:10.880179 (+ 315us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:41:10.882257 (+ 2078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:10.883685 (+ 1428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:10.885445 (+ 1760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:10.886215 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:10.886348 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:10.891408 (+ 5060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:41:10.891416 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:41:10.891431 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:41:10.891471 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:41:10.894780 (+ 3309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:41:10.895482 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:41:10.895495 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:41:10.895501 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:41:10.895682 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:41:10.896097 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:41:10.896103 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:41:10.896106 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:41:10.896877 (+ 771us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:41:10.897199 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0812 02:41:10.897744 (+ 545us) server_negotiation.cc:299] Negotiation successful
0812 02:41:10.898232 (+ 488us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":598,"thread_start_us":393,"threads_started":1}
I20260812 02:41:10.900256 28488 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9556e0ddfe724780a25219b08a2083b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9556e0ddfe724780a25219b08a2083b7" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 43367 } }
I20260812 02:41:10.900810 28488 leader_election.cc:302] T 00000000000000000000000000000000 P 9556e0ddfe724780a25219b08a2083b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9556e0ddfe724780a25219b08a2083b7; no voters:
I20260812 02:41:10.901588 28488 leader_election.cc:288] T 00000000000000000000000000000000 P 9556e0ddfe724780a25219b08a2083b7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 02:41:10.901733 28494 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9556e0ddfe724780a25219b08a2083b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:41:10.903688 28488 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9556e0ddfe724780a25219b08a2083b7 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:41:10.904614 28494 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9556e0ddfe724780a25219b08a2083b7 [term 1 LEADER]: Becoming Leader. State: Replica: 9556e0ddfe724780a25219b08a2083b7, State: Running, Role: LEADER
I20260812 02:41:10.905223 28494 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9556e0ddfe724780a25219b08a2083b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9556e0ddfe724780a25219b08a2083b7" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 43367 } }
I20260812 02:41:10.908434 28496 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9556e0ddfe724780a25219b08a2083b7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9556e0ddfe724780a25219b08a2083b7. Latest consensus state: current_term: 1 leader_uuid: "9556e0ddfe724780a25219b08a2083b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9556e0ddfe724780a25219b08a2083b7" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 43367 } } }
I20260812 02:41:10.908429 28495 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9556e0ddfe724780a25219b08a2083b7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9556e0ddfe724780a25219b08a2083b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9556e0ddfe724780a25219b08a2083b7" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 43367 } } }
I20260812 02:41:10.908867 28496 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9556e0ddfe724780a25219b08a2083b7 [sys.catalog]: This master's current role is: LEADER
I20260812 02:41:10.908876 28495 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9556e0ddfe724780a25219b08a2083b7 [sys.catalog]: This master's current role is: LEADER
I20260812 02:41:10.910611 28506 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:41:10.915194 28506 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 02:41:10.923589 28506 catalog_manager.cc:1347] Generated new cluster ID: 1e03099e7298441e8e17b9b894ed56ef
I20260812 02:41:10.923727 28506 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 02:41:10.951220 28506 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 02:41:10.953145 28506 catalog_manager.cc:1504] Loading token signing keys...
I20260812 02:41:10.965238 28506 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9556e0ddfe724780a25219b08a2083b7: Generated new TSK 0
I20260812 02:41:10.966428 28506 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.132.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.129.keytab.
Entry for principal kudu/127.7.132.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.129.keytab.
I20260812 02:41:11.049532 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502156971023-7698-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502156971023-7698-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502156971023-7698-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502156971023-7698-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.129:0
--local_ip_for_outbound_sockets=127.7.132.129
--webserver_interface=127.7.132.129
--webserver_port=0
--tserver_master_addrs=127.7.132.190:43367
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.129.keytab
--principal=kudu/127.7.132.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:45601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:41:11.404491 28518 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:41:11.404835 28518 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:41:11.405030 28518 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:41:11.412562 28518 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:41:11.412671 28518 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:41:11.412889 28518 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.129
I20260812 02:41:11.422881 28518 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:45601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502156971023-7698-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502156971023-7698-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.129.keytab
--principal=kudu/127.7.132.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502156971023-7698-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.132.129
--webserver_port=0
--tserver_master_addrs=127.7.132.190:43367
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.129
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502156971023-7698-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:41:11.424336 28518 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:41:11.425820 28518 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:41:11.440161 28528 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:41:11.440241 28531 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:41:11.440191 28529 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:41:11.440850 28518 server_base.cc:1034] running on GCE node
I20260812 02:41:11.441615 28518 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:41:11.442732 28518 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:41:11.443964 28518 hybrid_clock.cc:648] HybridClock initialized: now 1786502471443903 us; error 73 us; skew 500 ppm
I20260812 02:41:11.444298 28518 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:41:11 dist-test-slave-mn4r krb5kdc[28385](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502471, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:41:11.447810 28518 init.cc:377] Logged in from keytab as kudu/127.7.132.129@KRBTEST.COM (short username kudu)
I20260812 02:41:11.449425 28518 webserver.cc:460] Webserver started at http://127.7.132.129:34543/ using document root <none> and password file <none>
I20260812 02:41:11.450225 28518 fs_manager.cc:359] Metadata directory not provided
I20260812 02:41:11.450377 28518 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:41:11.450721 28518 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:41:11.453694 28518 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502156971023-7698-0/minicluster-data/ts-0/data/instance:
uuid: "5df304483391490eacd6964f31012980"
format_stamp: "Formatted at 2026-08-12 02:41:11 on dist-test-slave-mn4r"
server_key: "2111442fd8f8de8e2a07d978210d67d0"
server_key_iv: "ecdbbe56f282052c70b484fdf7353642"
server_key_version: "encryptionkey@0"
I20260812 02:41:11.454540 28518 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502156971023-7698-0/minicluster-data/ts-0/wal/instance:
uuid: "5df304483391490eacd6964f31012980"
format_stamp: "Formatted at 2026-08-12 02:41:11 on dist-test-slave-mn4r"
server_key: "2111442fd8f8de8e2a07d978210d67d0"
server_key_iv: "ecdbbe56f282052c70b484fdf7353642"
server_key_version: "encryptionkey@0"
I20260812 02:41:11.459807 28518 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 02:41:11.463378 28541 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:41:11.464931 28518 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260812 02:41:11.465095 28518 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502156971023-7698-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502156971023-7698-0/minicluster-data/ts-0/wal
uuid: "5df304483391490eacd6964f31012980"
format_stamp: "Formatted at 2026-08-12 02:41:11 on dist-test-slave-mn4r"
server_key: "2111442fd8f8de8e2a07d978210d67d0"
server_key_iv: "ecdbbe56f282052c70b484fdf7353642"
server_key_version: "encryptionkey@0"
I20260812 02:41:11.465324 28518 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502156971023-7698-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502156971023-7698-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502156971023-7698-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:41:11.486290 28518 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:41:11.490546 28518 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:41:11.490856 28518 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:41:11.491881 28518 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:41:11.493278 28518 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:41:11.493353 28518 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:41:11.493508 28518 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:41:11.493584 28518 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:41:11.530404 28518 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.129:37413
I20260812 02:41:11.530488 28676 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.129:37413 every 8 connection(s)
I20260812 02:41:11.531781 28518 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502156971023-7698-0/minicluster-data/ts-0/data/info.pb
I20260812 02:41:11.541188 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 28518
I20260812 02:41:11.541409 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502156971023-7698-0/minicluster-data/ts-0/wal/instance
I20260812 02:41:11.542291 7698 external_mini_cluster.cc:1428] Setting key 0b3b6e05f2d2f4a4002df3520b274dfa
Aug 12 02:41:11 dist-test-slave-mn4r krb5kdc[28385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502471, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.129@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 02:41:11.553465 28680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:11.534385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:50317 (local address 127.7.132.190:43367)
0812 02:41:11.534804 (+ 419us) server_negotiation.cc:206] Beginning negotiation
0812 02:41:11.534809 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:41:11.535876 (+ 1067us) server_negotiation.cc:407] Connection header received
0812 02:41:11.537784 (+ 1908us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:41:11.537790 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:41:11.537884 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:41:11.538004 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:41:11.539585 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:11.540322 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:11.542791 (+ 2469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:11.543256 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:11.543396 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:11.547553 (+ 4157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:41:11.547558 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:41:11.547561 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:41:11.547595 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:41:11.550224 (+ 2629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:41:11.551152 (+ 928us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:41:11.551156 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:41:11.551158 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:41:11.551270 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:41:11.551756 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:41:11.551759 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:41:11.551761 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:41:11.552008 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:41:11.552134 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0812 02:41:11.553038 (+ 904us) server_negotiation.cc:299] Negotiation successful
0812 02:41:11.553212 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":208,"threads_started":1}
I20260812 02:41:11.555085 28681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:11.535015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:43367 (local address 127.7.132.129:50317)
0812 02:41:11.535714 (+ 699us) negotiation.cc:107] Waiting for socket to connect
0812 02:41:11.535759 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 02:41:11.537505 (+ 1746us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:41:11.538275 (+ 770us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:41:11.538283 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:41:11.538824 (+ 541us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:41:11.539402 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:41:11.539421 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:41:11.540490 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:41:11.540494 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:41:11.542645 (+ 2151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:41:11.542661 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:41:11.543519 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:41:11.543527 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:41:11.543814 (+ 287us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:11.544544 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:41:11.544574 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:41:11.547316 (+ 2742us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:41:11.550356 (+ 3040us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:41:11.550368 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:41:11.550387 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:41:11.551029 (+ 642us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:41:11.551371 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:41:11.551378 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:41:11.551384 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:41:11.551644 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:41:11.552133 (+ 489us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:41:11.552145 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:41:11.552632 (+ 487us) client_negotiation.cc:769] Sending connection context
0812 02:41:11.552947 (+ 315us) client_negotiation.cc:240] Negotiation successful
0812 02:41:11.553557 (+ 610us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":269,"threads_started":1}
I20260812 02:41:11.556663 28677 heartbeater.cc:347] Connected to a master server at 127.7.132.190:43367
I20260812 02:41:11.556967 28677 heartbeater.cc:464] Registering TS with master...
I20260812 02:41:11.557813 28677 heartbeater.cc:511] Master 127.7.132.190:43367 requested a full tablet report, sending...
I20260812 02:41:11.560232 28442 ts_manager.cc:194] Registered new tserver with Master: 5df304483391490eacd6964f31012980 (127.7.132.129:37413)
I20260812 02:41:11.561714 28442 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.129@KRBTEST.COM'} at 127.7.132.129:50317
WARNING: no policy specified for kudu/127.7.132.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.130.keytab.
Entry for principal kudu/127.7.132.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.130.keytab.
I20260812 02:41:11.610626 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502156971023-7698-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502156971023-7698-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502156971023-7698-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502156971023-7698-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.130:0
--local_ip_for_outbound_sockets=127.7.132.130
--webserver_interface=127.7.132.130
--webserver_port=0
--tserver_master_addrs=127.7.132.190:43367
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.130.keytab
--principal=kudu/127.7.132.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:45601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:41:11.952703 28685 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:41:11.952994 28685 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:41:11.953187 28685 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:41:11.960474 28685 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:41:11.960577 28685 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:41:11.960786 28685 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.130
I20260812 02:41:11.970850 28685 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:45601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502156971023-7698-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502156971023-7698-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.130.keytab
--principal=kudu/127.7.132.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502156971023-7698-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.132.130
--webserver_port=0
--tserver_master_addrs=127.7.132.190:43367
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.130
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502156971023-7698-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:41:11.972162 28685 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:41:11.973519 28685 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:41:11.987146 28691 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:41:11.987097 28692 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:41:11.987182 28694 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:41:11.987911 28685 server_base.cc:1034] running on GCE node
I20260812 02:41:11.988595 28685 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:41:11.989581 28685 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:41:11.990823 28685 hybrid_clock.cc:648] HybridClock initialized: now 1786502471990766 us; error 46 us; skew 500 ppm
I20260812 02:41:11.991170 28685 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:41:11 dist-test-slave-mn4r krb5kdc[28385](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502471, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:41:11.994498 28685 init.cc:377] Logged in from keytab as kudu/127.7.132.130@KRBTEST.COM (short username kudu)
I20260812 02:41:11.995971 28685 webserver.cc:460] Webserver started at http://127.7.132.130:45687/ using document root <none> and password file <none>
I20260812 02:41:11.996703 28685 fs_manager.cc:359] Metadata directory not provided
I20260812 02:41:11.996857 28685 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:41:11.997191 28685 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:41:12.000226 28685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502156971023-7698-0/minicluster-data/ts-1/data/instance:
uuid: "61ef4fcfcf624a1fac61ccd107c00d41"
format_stamp: "Formatted at 2026-08-12 02:41:11 on dist-test-slave-mn4r"
server_key: "f2bc856bb8b2c1c6ac8d2680c33de00b"
server_key_iv: "d8a4437a27a9d5a3e02d562350368ad6"
server_key_version: "encryptionkey@0"
I20260812 02:41:12.001052 28685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502156971023-7698-0/minicluster-data/ts-1/wal/instance:
uuid: "61ef4fcfcf624a1fac61ccd107c00d41"
format_stamp: "Formatted at 2026-08-12 02:41:11 on dist-test-slave-mn4r"
server_key: "f2bc856bb8b2c1c6ac8d2680c33de00b"
server_key_iv: "d8a4437a27a9d5a3e02d562350368ad6"
server_key_version: "encryptionkey@0"
I20260812 02:41:12.006567 28685 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260812 02:41:12.010308 28701 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:41:12.011690 28685 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 02:41:12.011875 28685 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502156971023-7698-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502156971023-7698-0/minicluster-data/ts-1/wal
uuid: "61ef4fcfcf624a1fac61ccd107c00d41"
format_stamp: "Formatted at 2026-08-12 02:41:11 on dist-test-slave-mn4r"
server_key: "f2bc856bb8b2c1c6ac8d2680c33de00b"
server_key_iv: "d8a4437a27a9d5a3e02d562350368ad6"
server_key_version: "encryptionkey@0"
I20260812 02:41:12.012084 28685 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502156971023-7698-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502156971023-7698-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502156971023-7698-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:41:12.034930 28685 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:41:12.039516 28685 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:41:12.039878 28685 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:41:12.040980 28685 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:41:12.042467 28685 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:41:12.042574 28685 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:41:12.042730 28685 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:41:12.042805 28685 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:41:12.077704 28685 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.130:45993
I20260812 02:41:12.077757 28830 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.130:45993 every 8 connection(s)
I20260812 02:41:12.079129 28685 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502156971023-7698-0/minicluster-data/ts-1/data/info.pb
I20260812 02:41:12.080940 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 28685
I20260812 02:41:12.081156 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502156971023-7698-0/minicluster-data/ts-1/wal/instance
I20260812 02:41:12.081615 7698 external_mini_cluster.cc:1428] Setting key d896af419298ebec86a70caae917ca21
Aug 12 02:41:12 dist-test-slave-mn4r krb5kdc[28385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502471, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.130@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 02:41:12.103873 28834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:12.081929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:54259 (local address 127.7.132.190:43367)
0812 02:41:12.082741 (+ 812us) server_negotiation.cc:206] Beginning negotiation
0812 02:41:12.082749 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:41:12.083737 (+ 988us) server_negotiation.cc:407] Connection header received
0812 02:41:12.086843 (+ 3106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:41:12.086847 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:41:12.086913 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:41:12.086999 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:41:12.089206 (+ 2207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:12.089926 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:12.092471 (+ 2545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:12.092895 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:12.092988 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:12.097286 (+ 4298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:41:12.097292 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:41:12.097295 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:41:12.097356 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:41:12.100604 (+ 3248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:41:12.101357 (+ 753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:41:12.101363 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:41:12.101368 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:41:12.101502 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:41:12.101981 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:41:12.101990 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:41:12.101992 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:41:12.102338 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:41:12.102525 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0812 02:41:12.103229 (+ 704us) server_negotiation.cc:299] Negotiation successful
0812 02:41:12.103483 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":686,"thread_start_us":545,"threads_started":1}
I20260812 02:41:12.105007 28835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:12.082717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:43367 (local address 127.7.132.130:54259)
0812 02:41:12.083594 (+ 877us) negotiation.cc:107] Waiting for socket to connect
0812 02:41:12.083646 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0812 02:41:12.086509 (+ 2863us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:41:12.087290 (+ 781us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:41:12.087303 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:41:12.088157 (+ 854us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:41:12.089015 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:41:12.089044 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:41:12.090119 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:41:12.090127 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:41:12.092328 (+ 2201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:41:12.092347 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:41:12.093128 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:41:12.093137 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:41:12.093463 (+ 326us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:12.094268 (+ 805us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:41:12.094302 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:41:12.097021 (+ 2719us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:41:12.100749 (+ 3728us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:41:12.100758 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:41:12.100775 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:41:12.101222 (+ 447us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:41:12.101622 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:41:12.101627 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:41:12.101631 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:41:12.101826 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:41:12.102478 (+ 652us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:41:12.102488 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:41:12.102858 (+ 370us) client_negotiation.cc:769] Sending connection context
0812 02:41:12.103132 (+ 274us) client_negotiation.cc:240] Negotiation successful
0812 02:41:12.103698 (+ 566us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":553,"thread_start_us":336,"threads_started":1}
I20260812 02:41:12.106686 28831 heartbeater.cc:347] Connected to a master server at 127.7.132.190:43367
I20260812 02:41:12.107038 28831 heartbeater.cc:464] Registering TS with master...
I20260812 02:41:12.107887 28831 heartbeater.cc:511] Master 127.7.132.190:43367 requested a full tablet report, sending...
I20260812 02:41:12.109432 28442 ts_manager.cc:194] Registered new tserver with Master: 61ef4fcfcf624a1fac61ccd107c00d41 (127.7.132.130:45993)
I20260812 02:41:12.110343 28442 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.130@KRBTEST.COM'} at 127.7.132.130:54259
WARNING: no policy specified for kudu/127.7.132.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.131.keytab.
Entry for principal kudu/127.7.132.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.131.keytab.
I20260812 02:41:12.152783 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502156971023-7698-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502156971023-7698-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502156971023-7698-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502156971023-7698-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.131:0
--local_ip_for_outbound_sockets=127.7.132.131
--webserver_interface=127.7.132.131
--webserver_port=0
--tserver_master_addrs=127.7.132.190:43367
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.131.keytab
--principal=kudu/127.7.132.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:45601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:41:12.495306 28838 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:41:12.495601 28838 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:41:12.495769 28838 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:41:12.503199 28838 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:41:12.503381 28838 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:41:12.503516 28838 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.131
I20260812 02:41:12.515136 28838 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:45601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502156971023-7698-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502156971023-7698-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.131.keytab
--principal=kudu/127.7.132.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502156971023-7698-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.132.131
--webserver_port=0
--tserver_master_addrs=127.7.132.190:43367
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.131
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502156971023-7698-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:41:12.516486 28838 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:41:12.517882 28838 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:41:12.531574 28844 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:41:12.531548 28843 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:41:12.532546 28838 server_base.cc:1034] running on GCE node
W20260812 02:41:12.532992 28847 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:41:12.533787 28838 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:41:12.534989 28838 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:41:12.536275 28838 hybrid_clock.cc:648] HybridClock initialized: now 1786502472536213 us; error 52 us; skew 500 ppm
I20260812 02:41:12.536586 28838 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:41:12 dist-test-slave-mn4r krb5kdc[28385](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502472, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:41:12.539783 28838 init.cc:377] Logged in from keytab as kudu/127.7.132.131@KRBTEST.COM (short username kudu)
I20260812 02:41:12.541263 28838 webserver.cc:460] Webserver started at http://127.7.132.131:37283/ using document root <none> and password file <none>
I20260812 02:41:12.541978 28838 fs_manager.cc:359] Metadata directory not provided
I20260812 02:41:12.542169 28838 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:41:12.542508 28838 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:41:12.545486 28838 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502156971023-7698-0/minicluster-data/ts-2/data/instance:
uuid: "77d3c84d900f4d1199b9a0c4ea5fa19c"
format_stamp: "Formatted at 2026-08-12 02:41:12 on dist-test-slave-mn4r"
server_key: "8152a74ac7e3b8e0fd6e5f2e1d4c6d2a"
server_key_iv: "d7bda4a4f0d27863654ce2903182ab66"
server_key_version: "encryptionkey@0"
I20260812 02:41:12.546368 28838 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502156971023-7698-0/minicluster-data/ts-2/wal/instance:
uuid: "77d3c84d900f4d1199b9a0c4ea5fa19c"
format_stamp: "Formatted at 2026-08-12 02:41:12 on dist-test-slave-mn4r"
server_key: "8152a74ac7e3b8e0fd6e5f2e1d4c6d2a"
server_key_iv: "d7bda4a4f0d27863654ce2903182ab66"
server_key_version: "encryptionkey@0"
I20260812 02:41:12.551458 28838 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260812 02:41:12.554970 28855 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:41:12.556293 28838 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 02:41:12.556475 28838 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502156971023-7698-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502156971023-7698-0/minicluster-data/ts-2/wal
uuid: "77d3c84d900f4d1199b9a0c4ea5fa19c"
format_stamp: "Formatted at 2026-08-12 02:41:12 on dist-test-slave-mn4r"
server_key: "8152a74ac7e3b8e0fd6e5f2e1d4c6d2a"
server_key_iv: "d7bda4a4f0d27863654ce2903182ab66"
server_key_version: "encryptionkey@0"
I20260812 02:41:12.556676 28838 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502156971023-7698-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502156971023-7698-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502156971023-7698-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:41:12.565496 28677 heartbeater.cc:503] Master 127.7.132.190:43367 was elected leader, sending a full tablet report...
I20260812 02:41:12.590016 28838 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:41:12.594617 28838 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:41:12.594913 28838 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:41:12.595891 28838 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:41:12.597240 28838 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:41:12.597316 28838 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:41:12.597496 28838 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:41:12.597553 28838 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:41:12.631862 28838 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.131:42053
I20260812 02:41:12.631906 28986 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.131:42053 every 8 connection(s)
I20260812 02:41:12.633311 28838 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502156971023-7698-0/minicluster-data/ts-2/data/info.pb
I20260812 02:41:12.643370 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 28838
I20260812 02:41:12.643558 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502156971023-7698-0/minicluster-data/ts-2/wal/instance
I20260812 02:41:12.644027 7698 external_mini_cluster.cc:1428] Setting key ab788d60edc992cad744750437664700
Aug 12 02:41:12 dist-test-slave-mn4r krb5kdc[28385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502472, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.131@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 02:41:12.653628 28989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:12.635950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:56731 (local address 127.7.132.190:43367)
0812 02:41:12.636407 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0812 02:41:12.636412 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:41:12.637482 (+ 1070us) server_negotiation.cc:407] Connection header received
0812 02:41:12.639336 (+ 1854us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:41:12.639341 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:41:12.639408 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:41:12.639511 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:41:12.640951 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:12.641743 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:12.643343 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:12.643868 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:12.644006 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:12.647977 (+ 3971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:41:12.647984 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:41:12.647990 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:41:12.648036 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:41:12.650560 (+ 2524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:41:12.651300 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:41:12.651306 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:41:12.651310 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:41:12.651439 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:41:12.651869 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:41:12.651874 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:41:12.651878 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:41:12.652156 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:41:12.652345 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0812 02:41:12.653006 (+ 661us) server_negotiation.cc:299] Negotiation successful
0812 02:41:12.653242 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":327,"thread_start_us":225,"threads_started":1}
I20260812 02:41:12.654632 28990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:12.636605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:43367 (local address 127.7.132.131:56731)
0812 02:41:12.637299 (+ 694us) negotiation.cc:107] Waiting for socket to connect
0812 02:41:12.637345 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 02:41:12.639087 (+ 1742us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:41:12.639728 (+ 641us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:41:12.639735 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:41:12.640235 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:41:12.640795 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:41:12.640813 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:41:12.641874 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:41:12.641878 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:41:12.643189 (+ 1311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:41:12.643201 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:41:12.644123 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:41:12.644132 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:41:12.644365 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:12.645121 (+ 756us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:41:12.645149 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:41:12.647767 (+ 2618us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:41:12.650700 (+ 2933us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:41:12.650709 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:41:12.650725 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:41:12.651168 (+ 443us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:41:12.651556 (+ 388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:41:12.651561 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:41:12.651565 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:41:12.651747 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:41:12.652294 (+ 547us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:41:12.652303 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:41:12.652655 (+ 352us) client_negotiation.cc:769] Sending connection context
0812 02:41:12.652916 (+ 261us) client_negotiation.cc:240] Negotiation successful
0812 02:41:12.653430 (+ 514us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":281,"threads_started":1}
I20260812 02:41:12.656023 28987 heartbeater.cc:347] Connected to a master server at 127.7.132.190:43367
I20260812 02:41:12.656327 28987 heartbeater.cc:464] Registering TS with master...
I20260812 02:41:12.657042 28987 heartbeater.cc:511] Master 127.7.132.190:43367 requested a full tablet report, sending...
I20260812 02:41:12.658771 28441 ts_manager.cc:194] Registered new tserver with Master: 77d3c84d900f4d1199b9a0c4ea5fa19c (127.7.132.131:42053)
I20260812 02:41:12.659798 7698 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 02:41:12.660046 28441 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.131@KRBTEST.COM'} at 127.7.132.131:56731
I20260812 02:41:12.678967 28989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:12.665078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43050 (local address 127.7.132.190:43367)
0812 02:41:12.665296 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0812 02:41:12.665305 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 02:41:12.665671 (+ 366us) server_negotiation.cc:407] Connection header received
0812 02:41:12.665805 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:41:12.665809 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:41:12.665867 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:41:12.665943 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:41:12.667367 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:12.668085 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:12.669388 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:12.669973 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:12.670130 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:12.672862 (+ 2732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:41:12.672870 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:41:12.672876 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:41:12.672975 (+ 99us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:41:12.676084 (+ 3109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:41:12.676907 (+ 823us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:41:12.676912 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:41:12.676915 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:41:12.677051 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:41:12.677454 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:41:12.677458 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:41:12.677460 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:41:12.677828 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:41:12.678025 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0812 02:41:12.678543 (+ 518us) server_negotiation.cc:299] Negotiation successful
0812 02:41:12.678727 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 02:41:12.683296 28441 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43050:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 02:41:12.690277 28441 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 02:41:12.731141 29004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:12.702976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.130:45993 (local address 127.0.0.1:40620)
0812 02:41:12.720758 (+ 17782us) negotiation.cc:107] Waiting for socket to connect
0812 02:41:12.720782 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0812 02:41:12.721604 (+ 822us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:41:12.722646 (+ 1042us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:41:12.722655 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:41:12.722693 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:41:12.723147 (+ 454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:41:12.723161 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:41:12.725602 (+ 2441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:41:12.725611 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:41:12.728663 (+ 3052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:41:12.728677 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:41:12.729954 (+ 1277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:41:12.729969 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:41:12.730191 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:12.730241 (+ 50us) client_negotiation.cc:769] Sending connection context
0812 02:41:12.730403 (+ 162us) client_negotiation.cc:240] Negotiation successful
0812 02:41:12.730632 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17627,"mutex_wait_us":28,"spinlock_wait_cycles":1443712,"thread_start_us":15989,"threads_started":1}
I20260812 02:41:12.731328 29005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:12.706397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50848 (local address 127.7.132.131:42053)
0812 02:41:12.720228 (+ 13831us) server_negotiation.cc:206] Beginning negotiation
0812 02:41:12.720241 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 02:41:12.722632 (+ 2391us) server_negotiation.cc:407] Connection header received
0812 02:41:12.722871 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:41:12.722878 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:41:12.723147 (+ 269us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:41:12.723342 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:41:12.724241 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:12.725830 (+ 1589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:12.728388 (+ 2558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:12.729788 (+ 1400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:12.729922 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:12.730180 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0812 02:41:12.730796 (+ 616us) server_negotiation.cc:299] Negotiation successful
0812 02:41:12.730974 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13628,"thread_start_us":13471,"threads_started":1}
I20260812 02:41:12.731765 29001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:12.702926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56078 (local address 127.7.132.129:37413)
0812 02:41:12.722750 (+ 19824us) server_negotiation.cc:206] Beginning negotiation
0812 02:41:12.722759 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 02:41:12.722784 (+ 25us) server_negotiation.cc:407] Connection header received
0812 02:41:12.722882 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:41:12.722887 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:41:12.723063 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:41:12.723266 (+ 203us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:41:12.723916 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:12.725551 (+ 1635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:12.728756 (+ 3205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:12.730235 (+ 1479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:12.730373 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:12.730585 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0812 02:41:12.731255 (+ 670us) server_negotiation.cc:299] Negotiation successful
0812 02:41:12.731421 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19645,"thread_start_us":19453,"threads_started":1}
I20260812 02:41:12.731335 29000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:12.705538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.131:42053 (local address 127.0.0.1:50848)
0812 02:41:12.722325 (+ 16787us) negotiation.cc:107] Waiting for socket to connect
0812 02:41:12.722345 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 02:41:12.722471 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:41:12.723563 (+ 1092us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:41:12.723568 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:41:12.723596 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:41:12.724068 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:41:12.724078 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:41:12.726134 (+ 2056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:41:12.726139 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:41:12.728228 (+ 2089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:41:12.728241 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:41:12.730362 (+ 2121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:41:12.730368 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:41:12.730585 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:12.730625 (+ 40us) client_negotiation.cc:769] Sending connection context
0812 02:41:12.730739 (+ 114us) client_negotiation.cc:240] Negotiation successful
0812 02:41:12.730864 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16636,"mutex_wait_us":35,"thread_start_us":14885,"threads_started":1}
I20260812 02:41:12.732033 29003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:12.703016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40620 (local address 127.7.132.130:45993)
0812 02:41:12.721512 (+ 18496us) server_negotiation.cc:206] Beginning negotiation
0812 02:41:12.721521 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 02:41:12.721554 (+ 33us) server_negotiation.cc:407] Connection header received
0812 02:41:12.721977 (+ 423us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:41:12.721990 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:41:12.722384 (+ 394us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:41:12.723345 (+ 961us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:41:12.723622 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:12.725427 (+ 1805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:12.728830 (+ 3403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:12.729743 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:12.729902 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:12.730144 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0812 02:41:12.731574 (+ 1430us) server_negotiation.cc:299] Negotiation successful
0812 02:41:12.731726 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18320,"thread_start_us":18202,"threads_started":1}
I20260812 02:41:12.732435 29002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:12.702571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:37413 (local address 127.0.0.1:56078)
0812 02:41:12.719515 (+ 16944us) negotiation.cc:107] Waiting for socket to connect
0812 02:41:12.719586 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0812 02:41:12.721575 (+ 1989us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:41:12.723246 (+ 1671us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:41:12.723251 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:41:12.723278 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:41:12.723680 (+ 402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:41:12.723716 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:41:12.725715 (+ 1999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:41:12.725721 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:41:12.728573 (+ 2852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:41:12.728594 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:41:12.730507 (+ 1913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:41:12.730514 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:41:12.730683 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:12.730713 (+ 30us) client_negotiation.cc:769] Sending connection context
0812 02:41:12.730813 (+ 100us) client_negotiation.cc:240] Negotiation successful
0812 02:41:12.730939 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16567,"mutex_wait_us":33,"thread_start_us":19449,"threads_started":1}
I20260812 02:41:12.735668 28911 tablet_service.cc:1467] Processing CreateTablet for tablet 574c46b2897d48e0ab6a03cbc2787ce7 (DEFAULT_TABLE table=test-table [id=ca3f40cce3ed47ae95f43d25b11da5ad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:41:12.735812 28757 tablet_service.cc:1467] Processing CreateTablet for tablet 574c46b2897d48e0ab6a03cbc2787ce7 (DEFAULT_TABLE table=test-table [id=ca3f40cce3ed47ae95f43d25b11da5ad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:41:12.735816 28597 tablet_service.cc:1467] Processing CreateTablet for tablet 574c46b2897d48e0ab6a03cbc2787ce7 (DEFAULT_TABLE table=test-table [id=ca3f40cce3ed47ae95f43d25b11da5ad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:41:12.737174 28911 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 574c46b2897d48e0ab6a03cbc2787ce7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:41:12.737180 28597 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 574c46b2897d48e0ab6a03cbc2787ce7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:41:12.737179 28757 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 574c46b2897d48e0ab6a03cbc2787ce7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:41:12.748436 29006 tablet_bootstrap.cc:492] T 574c46b2897d48e0ab6a03cbc2787ce7 P 77d3c84d900f4d1199b9a0c4ea5fa19c: Bootstrap starting.
I20260812 02:41:12.751694 29008 tablet_bootstrap.cc:492] T 574c46b2897d48e0ab6a03cbc2787ce7 P 61ef4fcfcf624a1fac61ccd107c00d41: Bootstrap starting.
I20260812 02:41:12.752085 29006 tablet_bootstrap.cc:654] T 574c46b2897d48e0ab6a03cbc2787ce7 P 77d3c84d900f4d1199b9a0c4ea5fa19c: Neither blocks nor log segments found. Creating new log.
I20260812 02:41:12.752506 29007 tablet_bootstrap.cc:492] T 574c46b2897d48e0ab6a03cbc2787ce7 P 5df304483391490eacd6964f31012980: Bootstrap starting.
I20260812 02:41:12.753566 29006 log.cc:824] T 574c46b2897d48e0ab6a03cbc2787ce7 P 77d3c84d900f4d1199b9a0c4ea5fa19c: Log is configured to *not* fsync() on all Append() calls
I20260812 02:41:12.756096 29008 tablet_bootstrap.cc:654] T 574c46b2897d48e0ab6a03cbc2787ce7 P 61ef4fcfcf624a1fac61ccd107c00d41: Neither blocks nor log segments found. Creating new log.
I20260812 02:41:12.756141 29007 tablet_bootstrap.cc:654] T 574c46b2897d48e0ab6a03cbc2787ce7 P 5df304483391490eacd6964f31012980: Neither blocks nor log segments found. Creating new log.
I20260812 02:41:12.757218 29008 log.cc:824] T 574c46b2897d48e0ab6a03cbc2787ce7 P 61ef4fcfcf624a1fac61ccd107c00d41: Log is configured to *not* fsync() on all Append() calls
I20260812 02:41:12.757405 29007 log.cc:824] T 574c46b2897d48e0ab6a03cbc2787ce7 P 5df304483391490eacd6964f31012980: Log is configured to *not* fsync() on all Append() calls
I20260812 02:41:12.758029 29006 tablet_bootstrap.cc:492] T 574c46b2897d48e0ab6a03cbc2787ce7 P 77d3c84d900f4d1199b9a0c4ea5fa19c: No bootstrap required, opened a new log
I20260812 02:41:12.758399 29006 ts_tablet_manager.cc:1397] T 574c46b2897d48e0ab6a03cbc2787ce7 P 77d3c84d900f4d1199b9a0c4ea5fa19c: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.006s
I20260812 02:41:12.760130 29008 tablet_bootstrap.cc:492] T 574c46b2897d48e0ab6a03cbc2787ce7 P 61ef4fcfcf624a1fac61ccd107c00d41: No bootstrap required, opened a new log
I20260812 02:41:12.760385 29008 ts_tablet_manager.cc:1397] T 574c46b2897d48e0ab6a03cbc2787ce7 P 61ef4fcfcf624a1fac61ccd107c00d41: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260812 02:41:12.764240 29007 tablet_bootstrap.cc:492] T 574c46b2897d48e0ab6a03cbc2787ce7 P 5df304483391490eacd6964f31012980: No bootstrap required, opened a new log
I20260812 02:41:12.764524 29007 ts_tablet_manager.cc:1397] T 574c46b2897d48e0ab6a03cbc2787ce7 P 5df304483391490eacd6964f31012980: Time spent bootstrapping tablet: real 0.012s user 0.000s sys 0.007s
I20260812 02:41:12.767803 29008 raft_consensus.cc:348] T 574c46b2897d48e0ab6a03cbc2787ce7 P 61ef4fcfcf624a1fac61ccd107c00d41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5df304483391490eacd6964f31012980" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 37413 } } peers { permanent_uuid: "61ef4fcfcf624a1fac61ccd107c00d41" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 45993 } } peers { permanent_uuid: "77d3c84d900f4d1199b9a0c4ea5fa19c" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 42053 } }
I20260812 02:41:12.768194 29008 raft_consensus.cc:374] T 574c46b2897d48e0ab6a03cbc2787ce7 P 61ef4fcfcf624a1fac61ccd107c00d41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:41:12.767988 29006 raft_consensus.cc:348] T 574c46b2897d48e0ab6a03cbc2787ce7 P 77d3c84d900f4d1199b9a0c4ea5fa19c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5df304483391490eacd6964f31012980" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 37413 } } peers { permanent_uuid: "61ef4fcfcf624a1fac61ccd107c00d41" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 45993 } } peers { permanent_uuid: "77d3c84d900f4d1199b9a0c4ea5fa19c" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 42053 } }
I20260812 02:41:12.768301 29008 raft_consensus.cc:729] T 574c46b2897d48e0ab6a03cbc2787ce7 P 61ef4fcfcf624a1fac61ccd107c00d41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61ef4fcfcf624a1fac61ccd107c00d41, State: Initialized, Role: FOLLOWER
I20260812 02:41:12.768406 29006 raft_consensus.cc:374] T 574c46b2897d48e0ab6a03cbc2787ce7 P 77d3c84d900f4d1199b9a0c4ea5fa19c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:41:12.768501 29006 raft_consensus.cc:729] T 574c46b2897d48e0ab6a03cbc2787ce7 P 77d3c84d900f4d1199b9a0c4ea5fa19c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77d3c84d900f4d1199b9a0c4ea5fa19c, State: Initialized, Role: FOLLOWER
I20260812 02:41:12.769172 29008 consensus_queue.cc:260] T 574c46b2897d48e0ab6a03cbc2787ce7 P 61ef4fcfcf624a1fac61ccd107c00d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5df304483391490eacd6964f31012980" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 37413 } } peers { permanent_uuid: "61ef4fcfcf624a1fac61ccd107c00d41" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 45993 } } peers { permanent_uuid: "77d3c84d900f4d1199b9a0c4ea5fa19c" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 42053 } }
I20260812 02:41:12.769865 29006 consensus_queue.cc:260] T 574c46b2897d48e0ab6a03cbc2787ce7 P 77d3c84d900f4d1199b9a0c4ea5fa19c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5df304483391490eacd6964f31012980" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 37413 } } peers { permanent_uuid: "61ef4fcfcf624a1fac61ccd107c00d41" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 45993 } } peers { permanent_uuid: "77d3c84d900f4d1199b9a0c4ea5fa19c" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 42053 } }
I20260812 02:41:12.771235 28987 heartbeater.cc:503] Master 127.7.132.190:43367 was elected leader, sending a full tablet report...
I20260812 02:41:12.774817 28831 heartbeater.cc:503] Master 127.7.132.190:43367 was elected leader, sending a full tablet report...
I20260812 02:41:12.774911 29007 raft_consensus.cc:348] T 574c46b2897d48e0ab6a03cbc2787ce7 P 5df304483391490eacd6964f31012980 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5df304483391490eacd6964f31012980" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 37413 } } peers { permanent_uuid: "61ef4fcfcf624a1fac61ccd107c00d41" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 45993 } } peers { permanent_uuid: "77d3c84d900f4d1199b9a0c4ea5fa19c" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 42053 } }
I20260812 02:41:12.775357 29007 raft_consensus.cc:374] T 574c46b2897d48e0ab6a03cbc2787ce7 P 5df304483391490eacd6964f31012980 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:41:12.775453 29007 raft_consensus.cc:729] T 574c46b2897d48e0ab6a03cbc2787ce7 P 5df304483391490eacd6964f31012980 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5df304483391490eacd6964f31012980, State: Initialized, Role: FOLLOWER
I20260812 02:41:12.775501 29008 ts_tablet_manager.cc:1428] T 574c46b2897d48e0ab6a03cbc2787ce7 P 61ef4fcfcf624a1fac61ccd107c00d41: Time spent starting tablet: real 0.015s user 0.005s sys 0.010s
I20260812 02:41:12.776206 29006 ts_tablet_manager.cc:1428] T 574c46b2897d48e0ab6a03cbc2787ce7 P 77d3c84d900f4d1199b9a0c4ea5fa19c: Time spent starting tablet: real 0.018s user 0.011s sys 0.001s
I20260812 02:41:12.776140 29007 consensus_queue.cc:260] T 574c46b2897d48e0ab6a03cbc2787ce7 P 5df304483391490eacd6964f31012980 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5df304483391490eacd6964f31012980" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 37413 } } peers { permanent_uuid: "61ef4fcfcf624a1fac61ccd107c00d41" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 45993 } } peers { permanent_uuid: "77d3c84d900f4d1199b9a0c4ea5fa19c" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 42053 } }
I20260812 02:41:12.778241 29007 ts_tablet_manager.cc:1428] T 574c46b2897d48e0ab6a03cbc2787ce7 P 5df304483391490eacd6964f31012980: Time spent starting tablet: real 0.014s user 0.013s sys 0.001s
W20260812 02:41:12.784122 28678 tablet.cc:2369] T 574c46b2897d48e0ab6a03cbc2787ce7 P 5df304483391490eacd6964f31012980: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 02:41:12.831248 28832 tablet.cc:2369] T 574c46b2897d48e0ab6a03cbc2787ce7 P 61ef4fcfcf624a1fac61ccd107c00d41: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 02:41:12.885452 28988 tablet.cc:2369] T 574c46b2897d48e0ab6a03cbc2787ce7 P 77d3c84d900f4d1199b9a0c4ea5fa19c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 02:41:12.949577 29015 raft_consensus.cc:482] T 574c46b2897d48e0ab6a03cbc2787ce7 P 5df304483391490eacd6964f31012980 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 02:41:12.949870 29015 raft_consensus.cc:504] T 574c46b2897d48e0ab6a03cbc2787ce7 P 5df304483391490eacd6964f31012980 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5df304483391490eacd6964f31012980" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 37413 } } peers { permanent_uuid: "61ef4fcfcf624a1fac61ccd107c00d41" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 45993 } } peers { permanent_uuid: "77d3c84d900f4d1199b9a0c4ea5fa19c" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 42053 } }
I20260812 02:41:12.951738 29015 leader_election.cc:288] T 574c46b2897d48e0ab6a03cbc2787ce7 P 5df304483391490eacd6964f31012980 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 61ef4fcfcf624a1fac61ccd107c00d41 (127.7.132.130:45993), 77d3c84d900f4d1199b9a0c4ea5fa19c (127.7.132.131:42053)
I20260812 02:41:12.958878 29005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:12.952487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:45121 (local address 127.7.132.131:42053)
0812 02:41:12.952688 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0812 02:41:12.952696 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:41:12.953387 (+ 691us) server_negotiation.cc:407] Connection header received
0812 02:41:12.953657 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:41:12.953666 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:41:12.953726 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:41:12.953823 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:41:12.954541 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:12.955304 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:12.957172 (+ 1868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:12.957937 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:12.958134 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:12.958215 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 02:41:12.958488 (+ 273us) server_negotiation.cc:299] Negotiation successful
0812 02:41:12.958615 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 02:41:12.959060 29017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:12.952385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.131:42053 (local address 127.7.132.129:45121)
0812 02:41:12.953261 (+ 876us) negotiation.cc:107] Waiting for socket to connect
0812 02:41:12.953276 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 02:41:12.953508 (+ 232us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:41:12.953839 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:41:12.953843 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:41:12.953866 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:41:12.954376 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:41:12.954386 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:41:12.955441 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:41:12.955450 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:41:12.957014 (+ 1564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:41:12.957026 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:41:12.958174 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:41:12.958180 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:41:12.958324 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:12.958347 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 02:41:12.958461 (+ 114us) client_negotiation.cc:240] Negotiation successful
0812 02:41:12.958596 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":769,"mutex_wait_us":31,"spinlock_wait_cycles":113792,"thread_start_us":489,"threads_started":1}
I20260812 02:41:12.959343 29003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:12.952266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:57299 (local address 127.7.132.130:45993)
0812 02:41:12.952522 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0812 02:41:12.952533 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 02:41:12.953365 (+ 832us) server_negotiation.cc:407] Connection header received
0812 02:41:12.953644 (+ 279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:41:12.953648 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:41:12.953754 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:41:12.953867 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:41:12.954620 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:12.955406 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:12.957373 (+ 1967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:12.958318 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:12.958505 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:12.958592 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0812 02:41:12.958924 (+ 332us) server_negotiation.cc:299] Negotiation successful
0812 02:41:12.959063 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260812 02:41:12.959514 29016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:12.952001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.130:45993 (local address 127.7.132.129:57299)
0812 02:41:12.953222 (+ 1221us) negotiation.cc:107] Waiting for socket to connect
0812 02:41:12.953240 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 02:41:12.953473 (+ 233us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:41:12.953892 (+ 419us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:41:12.953896 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:41:12.953922 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:41:12.954457 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:41:12.954467 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:41:12.955571 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:41:12.955581 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:41:12.957194 (+ 1613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:41:12.957207 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:41:12.958560 (+ 1353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:41:12.958567 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:41:12.958743 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:12.958767 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 02:41:12.958890 (+ 123us) client_negotiation.cc:240] Negotiation successful
0812 02:41:12.959025 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1058,"mutex_wait_us":23,"thread_start_us":759,"threads_started":1}
I20260812 02:41:12.960501 28933 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "574c46b2897d48e0ab6a03cbc2787ce7" candidate_uuid: "5df304483391490eacd6964f31012980" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "77d3c84d900f4d1199b9a0c4ea5fa19c" is_pre_election: true
I20260812 02:41:12.960726 28779 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "574c46b2897d48e0ab6a03cbc2787ce7" candidate_uuid: "5df304483391490eacd6964f31012980" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61ef4fcfcf624a1fac61ccd107c00d41" is_pre_election: true
I20260812 02:41:12.961186 28933 raft_consensus.cc:2454] T 574c46b2897d48e0ab6a03cbc2787ce7 P 77d3c84d900f4d1199b9a0c4ea5fa19c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5df304483391490eacd6964f31012980 in term 0.
I20260812 02:41:12.961285 28779 raft_consensus.cc:2454] T 574c46b2897d48e0ab6a03cbc2787ce7 P 61ef4fcfcf624a1fac61ccd107c00d41 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5df304483391490eacd6964f31012980 in term 0.
I20260812 02:41:12.962121 28543 leader_election.cc:302] T 574c46b2897d48e0ab6a03cbc2787ce7 P 5df304483391490eacd6964f31012980 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 5df304483391490eacd6964f31012980, 61ef4fcfcf624a1fac61ccd107c00d41, 77d3c84d900f4d1199b9a0c4ea5fa19c; no voters:
I20260812 02:41:12.962628 29015 raft_consensus.cc:2781] T 574c46b2897d48e0ab6a03cbc2787ce7 P 5df304483391490eacd6964f31012980 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 02:41:12.962811 29015 raft_consensus.cc:482] T 574c46b2897d48e0ab6a03cbc2787ce7 P 5df304483391490eacd6964f31012980 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 02:41:12.963002 29015 raft_consensus.cc:3037] T 574c46b2897d48e0ab6a03cbc2787ce7 P 5df304483391490eacd6964f31012980 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:41:12.965309 29015 raft_consensus.cc:504] T 574c46b2897d48e0ab6a03cbc2787ce7 P 5df304483391490eacd6964f31012980 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5df304483391490eacd6964f31012980" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 37413 } } peers { permanent_uuid: "61ef4fcfcf624a1fac61ccd107c00d41" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 45993 } } peers { permanent_uuid: "77d3c84d900f4d1199b9a0c4ea5fa19c" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 42053 } }
I20260812 02:41:12.965930 29015 leader_election.cc:288] T 574c46b2897d48e0ab6a03cbc2787ce7 P 5df304483391490eacd6964f31012980 [CANDIDATE]: Term 1 election: Requested vote from peers 61ef4fcfcf624a1fac61ccd107c00d41 (127.7.132.130:45993), 77d3c84d900f4d1199b9a0c4ea5fa19c (127.7.132.131:42053)
I20260812 02:41:12.966487 28779 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "574c46b2897d48e0ab6a03cbc2787ce7" candidate_uuid: "5df304483391490eacd6964f31012980" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61ef4fcfcf624a1fac61ccd107c00d41"
I20260812 02:41:12.966475 28933 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "574c46b2897d48e0ab6a03cbc2787ce7" candidate_uuid: "5df304483391490eacd6964f31012980" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "77d3c84d900f4d1199b9a0c4ea5fa19c"
I20260812 02:41:12.966691 28933 raft_consensus.cc:3037] T 574c46b2897d48e0ab6a03cbc2787ce7 P 77d3c84d900f4d1199b9a0c4ea5fa19c [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:41:12.966691 28779 raft_consensus.cc:3037] T 574c46b2897d48e0ab6a03cbc2787ce7 P 61ef4fcfcf624a1fac61ccd107c00d41 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:41:12.969900 28779 raft_consensus.cc:2454] T 574c46b2897d48e0ab6a03cbc2787ce7 P 61ef4fcfcf624a1fac61ccd107c00d41 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5df304483391490eacd6964f31012980 in term 1.
I20260812 02:41:12.969900 28933 raft_consensus.cc:2454] T 574c46b2897d48e0ab6a03cbc2787ce7 P 77d3c84d900f4d1199b9a0c4ea5fa19c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5df304483391490eacd6964f31012980 in term 1.
I20260812 02:41:12.970451 28543 leader_election.cc:302] T 574c46b2897d48e0ab6a03cbc2787ce7 P 5df304483391490eacd6964f31012980 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5df304483391490eacd6964f31012980, 77d3c84d900f4d1199b9a0c4ea5fa19c; no voters:
I20260812 02:41:12.970883 29015 raft_consensus.cc:2781] T 574c46b2897d48e0ab6a03cbc2787ce7 P 5df304483391490eacd6964f31012980 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:41:12.971403 29015 raft_consensus.cc:686] T 574c46b2897d48e0ab6a03cbc2787ce7 P 5df304483391490eacd6964f31012980 [term 1 LEADER]: Becoming Leader. State: Replica: 5df304483391490eacd6964f31012980, State: Running, Role: LEADER
I20260812 02:41:12.971892 29015 consensus_queue.cc:237] T 574c46b2897d48e0ab6a03cbc2787ce7 P 5df304483391490eacd6964f31012980 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5df304483391490eacd6964f31012980" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 37413 } } peers { permanent_uuid: "61ef4fcfcf624a1fac61ccd107c00d41" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 45993 } } peers { permanent_uuid: "77d3c84d900f4d1199b9a0c4ea5fa19c" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 42053 } }
I20260812 02:41:12.976970 28441 catalog_manager.cc:5495] T 574c46b2897d48e0ab6a03cbc2787ce7 P 5df304483391490eacd6964f31012980 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5df304483391490eacd6964f31012980 (127.7.132.129). New cstate: current_term: 1 leader_uuid: "5df304483391490eacd6964f31012980" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5df304483391490eacd6964f31012980" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 37413 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "61ef4fcfcf624a1fac61ccd107c00d41" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 45993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "77d3c84d900f4d1199b9a0c4ea5fa19c" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 42053 } health_report { overall_health: UNKNOWN } } }
I20260812 02:41:13.013422 29001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:13.008048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56084 (local address 127.7.132.129:37413)
0812 02:41:13.008240 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0812 02:41:13.008245 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:41:13.008384 (+ 139us) server_negotiation.cc:407] Connection header received
0812 02:41:13.008602 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:41:13.008606 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:41:13.008690 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:41:13.008838 (+ 148us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 02:41:13.009189 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:13.009914 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:13.011400 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:13.011807 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:13.011901 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:13.012371 (+ 470us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 02:41:13.012924 (+ 553us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 02:41:13.013067 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0812 02:41:13.013147 (+ 80us) server_negotiation.cc:299] Negotiation successful
0812 02:41:13.013217 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260812 02:41:13.021790 28779 raft_consensus.cc:1261] T 574c46b2897d48e0ab6a03cbc2787ce7 P 61ef4fcfcf624a1fac61ccd107c00d41 [term 1 FOLLOWER]: Refusing update from remote peer 5df304483391490eacd6964f31012980: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:41:13.021785 28933 raft_consensus.cc:1261] T 574c46b2897d48e0ab6a03cbc2787ce7 P 77d3c84d900f4d1199b9a0c4ea5fa19c [term 1 FOLLOWER]: Refusing update from remote peer 5df304483391490eacd6964f31012980: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:41:13.022835 29018 consensus_queue.cc:1035] T 574c46b2897d48e0ab6a03cbc2787ce7 P 5df304483391490eacd6964f31012980 [LEADER]: Connected to new peer: Peer: permanent_uuid: "77d3c84d900f4d1199b9a0c4ea5fa19c" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 42053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:41:13.023268 29015 consensus_queue.cc:1035] T 574c46b2897d48e0ab6a03cbc2787ce7 P 5df304483391490eacd6964f31012980 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61ef4fcfcf624a1fac61ccd107c00d41" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 45993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:41:13.035614 29021 mvcc.cc:205] Tried to move back new op lower bound from 7317514129487597568 to 7317514129296707584. Current Snapshot: MvccSnapshot[applied={T|T < 7317514129487597568}]
I20260812 02:41:13.038628 29023 mvcc.cc:205] Tried to move back new op lower bound from 7317514129487597568 to 7317514129296707584. Current Snapshot: MvccSnapshot[applied={T|T < 7317514129487597568}]
I20260812 02:41:13.049278 28442 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43050:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 02:41:13.049580 28442 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43050:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 02:41:13.053514 28442 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9556e0ddfe724780a25219b08a2083b7: Sending DeleteTablet for 3 replicas of tablet 574c46b2897d48e0ab6a03cbc2787ce7
I20260812 02:41:13.054849 28757 tablet_service.cc:1514] Processing DeleteTablet for tablet 574c46b2897d48e0ab6a03cbc2787ce7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:41:13 UTC) from {username='kudu', principal='kudu/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:40620
I20260812 02:41:13.054860 28597 tablet_service.cc:1514] Processing DeleteTablet for tablet 574c46b2897d48e0ab6a03cbc2787ce7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:41:13 UTC) from {username='kudu', principal='kudu/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:56078
I20260812 02:41:13.055622 28911 tablet_service.cc:1514] Processing DeleteTablet for tablet 574c46b2897d48e0ab6a03cbc2787ce7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:41:13 UTC) from {username='kudu', principal='kudu/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:50848
I20260812 02:41:13.056188 29031 tablet_replica.cc:330] T 574c46b2897d48e0ab6a03cbc2787ce7 P 5df304483391490eacd6964f31012980: stopping tablet replica
I20260812 02:41:13.056730 29031 raft_consensus.cc:2229] T 574c46b2897d48e0ab6a03cbc2787ce7 P 5df304483391490eacd6964f31012980 [term 1 LEADER]: Raft consensus shutting down.
I20260812 02:41:13.057446 29031 raft_consensus.cc:2258] T 574c46b2897d48e0ab6a03cbc2787ce7 P 5df304483391490eacd6964f31012980 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:41:13.058377 29032 tablet_replica.cc:330] T 574c46b2897d48e0ab6a03cbc2787ce7 P 77d3c84d900f4d1199b9a0c4ea5fa19c: stopping tablet replica
I20260812 02:41:13.058796 29032 raft_consensus.cc:2229] T 574c46b2897d48e0ab6a03cbc2787ce7 P 77d3c84d900f4d1199b9a0c4ea5fa19c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 02:41:13.058935 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 28518
I20260812 02:41:13.059315 29032 raft_consensus.cc:2258] T 574c46b2897d48e0ab6a03cbc2787ce7 P 77d3c84d900f4d1199b9a0c4ea5fa19c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:41:13.063197 29032 ts_tablet_manager.cc:1905] T 574c46b2897d48e0ab6a03cbc2787ce7 P 77d3c84d900f4d1199b9a0c4ea5fa19c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:41:13.063221 29031 ts_tablet_manager.cc:1905] T 574c46b2897d48e0ab6a03cbc2787ce7 P 5df304483391490eacd6964f31012980: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:41:13.071695 29031 ts_tablet_manager.cc:1918] T 574c46b2897d48e0ab6a03cbc2787ce7 P 5df304483391490eacd6964f31012980: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 02:41:13.072355 29030 tablet_replica.cc:330] T 574c46b2897d48e0ab6a03cbc2787ce7 P 61ef4fcfcf624a1fac61ccd107c00d41: stopping tablet replica
I20260812 02:41:13.072861 29030 raft_consensus.cc:2229] T 574c46b2897d48e0ab6a03cbc2787ce7 P 61ef4fcfcf624a1fac61ccd107c00d41 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 02:41:13.073457 29030 raft_consensus.cc:2258] T 574c46b2897d48e0ab6a03cbc2787ce7 P 61ef4fcfcf624a1fac61ccd107c00d41 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:41:13.073904 29031 log.cc:1196] T 574c46b2897d48e0ab6a03cbc2787ce7 P 5df304483391490eacd6964f31012980: Deleting WAL directory at /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502156971023-7698-0/minicluster-data/ts-0/wal/wals/574c46b2897d48e0ab6a03cbc2787ce7
I20260812 02:41:13.074610 29031 ts_tablet_manager.cc:1939] T 574c46b2897d48e0ab6a03cbc2787ce7 P 5df304483391490eacd6964f31012980: Deleting consensus metadata
I20260812 02:41:13.076749 28423 catalog_manager.cc:4841] TS 5df304483391490eacd6964f31012980 (127.7.132.129:37413): tablet 574c46b2897d48e0ab6a03cbc2787ce7 (table test-table [id=ca3f40cce3ed47ae95f43d25b11da5ad]) successfully deleted
I20260812 02:41:13.077337 29032 ts_tablet_manager.cc:1918] T 574c46b2897d48e0ab6a03cbc2787ce7 P 77d3c84d900f4d1199b9a0c4ea5fa19c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 02:41:13.077598 29032 log.cc:1196] T 574c46b2897d48e0ab6a03cbc2787ce7 P 77d3c84d900f4d1199b9a0c4ea5fa19c: Deleting WAL directory at /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502156971023-7698-0/minicluster-data/ts-2/wal/wals/574c46b2897d48e0ab6a03cbc2787ce7
I20260812 02:41:13.078202 29032 ts_tablet_manager.cc:1939] T 574c46b2897d48e0ab6a03cbc2787ce7 P 77d3c84d900f4d1199b9a0c4ea5fa19c: Deleting consensus metadata
I20260812 02:41:13.080284 28421 catalog_manager.cc:4841] TS 77d3c84d900f4d1199b9a0c4ea5fa19c (127.7.132.131:42053): tablet 574c46b2897d48e0ab6a03cbc2787ce7 (table test-table [id=ca3f40cce3ed47ae95f43d25b11da5ad]) successfully deleted
I20260812 02:41:13.082897 29030 ts_tablet_manager.cc:1905] T 574c46b2897d48e0ab6a03cbc2787ce7 P 61ef4fcfcf624a1fac61ccd107c00d41: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 12 02:41:13 dist-test-slave-mn4r krb5kdc[28385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502470, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.132.129@KRBTEST.COM
I20260812 02:41:13.090811 29030 ts_tablet_manager.cc:1918] T 574c46b2897d48e0ab6a03cbc2787ce7 P 61ef4fcfcf624a1fac61ccd107c00d41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 02:41:13.091171 29030 log.cc:1196] T 574c46b2897d48e0ab6a03cbc2787ce7 P 61ef4fcfcf624a1fac61ccd107c00d41: Deleting WAL directory at /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786502156971023-7698-0/minicluster-data/ts-1/wal/wals/574c46b2897d48e0ab6a03cbc2787ce7
I20260812 02:41:13.091835 29030 ts_tablet_manager.cc:1939] T 574c46b2897d48e0ab6a03cbc2787ce7 P 61ef4fcfcf624a1fac61ccd107c00d41: Deleting consensus metadata
I20260812 02:41:13.093912 28422 catalog_manager.cc:4841] TS 61ef4fcfcf624a1fac61ccd107c00d41 (127.7.132.130:45993): tablet 574c46b2897d48e0ab6a03cbc2787ce7 (table test-table [id=ca3f40cce3ed47ae95f43d25b11da5ad]) successfully deleted
I20260812 02:41:13.094060 29001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:13.060120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56092 (local address 127.7.132.129:37413)
0812 02:41:13.060338 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0812 02:41:13.060346 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:41:13.071102 (+ 10756us) server_negotiation.cc:407] Connection header received
0812 02:41:13.071710 (+ 608us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:41:13.071719 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:41:13.071846 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:41:13.071922 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:41:13.074011 (+ 2089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:13.075292 (+ 1281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:13.078420 (+ 3128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:13.079224 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:13.079335 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:13.085546 (+ 6211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:41:13.085566 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:41:13.085586 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:41:13.085647 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:41:13.089937 (+ 4290us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:41:13.091063 (+ 1126us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:41:13.091073 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:41:13.091076 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:41:13.091305 (+ 229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:41:13.091884 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:41:13.091892 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:41:13.091896 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:41:13.092429 (+ 533us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:41:13.092729 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0812 02:41:13.093378 (+ 649us) server_negotiation.cc:299] Negotiation successful
0812 02:41:13.093655 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260812 02:41:13.095564 28672 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:41:13.196812 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 28518
I20260812 02:41:13.218890 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 28685
Aug 12 02:41:13 dist-test-slave-mn4r krb5kdc[28385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502470, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.132.130@KRBTEST.COM
I20260812 02:41:13.234512 29003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:13.219634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40622 (local address 127.7.132.130:45993)
0812 02:41:13.219804 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0812 02:41:13.219809 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:41:13.219883 (+ 74us) server_negotiation.cc:407] Connection header received
0812 02:41:13.220062 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:41:13.220066 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:41:13.220132 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:41:13.220219 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:41:13.221558 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:13.222413 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:13.223740 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:13.224182 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:13.224276 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:13.228829 (+ 4553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:41:13.228844 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:41:13.228858 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:41:13.228894 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:41:13.231695 (+ 2801us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:41:13.232457 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:41:13.232470 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:41:13.232476 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:41:13.232624 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:41:13.232974 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:41:13.232979 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:41:13.232982 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:41:13.233383 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:41:13.233614 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0812 02:41:13.233857 (+ 243us) server_negotiation.cc:299] Negotiation successful
0812 02:41:13.234017 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 02:41:13.235152 28826 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:41:13.323218 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 28685
I20260812 02:41:13.346024 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 28838
Aug 12 02:41:13 dist-test-slave-mn4r krb5kdc[28385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502470, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.132.131@KRBTEST.COM
I20260812 02:41:13.364012 29005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:13.347219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50860 (local address 127.7.132.131:42053)
0812 02:41:13.347461 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0812 02:41:13.347468 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:41:13.347492 (+ 24us) server_negotiation.cc:407] Connection header received
0812 02:41:13.347548 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:41:13.347552 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:41:13.347623 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:41:13.347736 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:41:13.349621 (+ 1885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:13.350559 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:13.352565 (+ 2006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:13.353008 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:13.353133 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:13.357654 (+ 4521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:41:13.357669 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:41:13.357685 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:41:13.357722 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:41:13.361085 (+ 3363us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:41:13.361822 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:41:13.361840 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:41:13.361842 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:41:13.362002 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:41:13.362423 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:41:13.362430 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:41:13.362438 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:41:13.362793 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:41:13.363105 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0812 02:41:13.363429 (+ 324us) server_negotiation.cc:299] Negotiation successful
0812 02:41:13.363649 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260812 02:41:13.364661 28980 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:41:13.454654 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 28838
I20260812 02:41:13.475405 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 28400
I20260812 02:41:13.476588 28483 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:41:13.588829 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 28400
2026-08-12T02:41:13Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (4772 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-12T02:41:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T02:41:13Z Disabled control of system clock
I20260812 02:41:13.688347 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502156971023-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502156971023-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502156971023-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:40969
--webserver_interface=127.7.132.190
--webserver_port=0
--builtin_ntp_servers=127.7.132.148:44733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.132.190:40969
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 02:41:14.037492 29041 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:41:14.037765 29041 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:41:14.037844 29041 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:41:14.045250 29041 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:41:14.045351 29041 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:41:14.045395 29041 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:41:14.045454 29041 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:41:14.045497 29041 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:41:14.054854 29041 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:44733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502156971023-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502156971023-7698-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.132.190:40969
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:40969
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.132.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502156971023-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:41:14.056142 29041 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:41:14.057976 29041 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:41:14.070417 29047 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:41:14.070533 29048 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:41:14.070449 29050 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:41:14.071666 29041 server_base.cc:1034] running on GCE node
I20260812 02:41:14.072407 29041 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:41:14.073835 29041 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:41:14.075047 29041 hybrid_clock.cc:648] HybridClock initialized: now 1786502474074963 us; error 66 us; skew 500 ppm
I20260812 02:41:14.075444 29041 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 02:41:14.084313 29041 webserver.cc:460] Webserver started at http://127.7.132.190:35337/ using document root <none> and password file <none>
I20260812 02:41:14.085134 29041 fs_manager.cc:359] Metadata directory not provided
I20260812 02:41:14.085284 29041 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:41:14.085646 29041 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:41:14.088846 29041 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502156971023-7698-0/minicluster-data/master-0/data/instance:
uuid: "23dea4a07da149b5b72720ae55fe1981"
format_stamp: "Formatted at 2026-08-12 02:41:14 on dist-test-slave-mn4r"
server_key: "2c97561b4421f121c4f015f0c7e148b9"
server_key_iv: "087900426d2b64426c1ad97962ce288f"
server_key_version: "encryptionkey@0"
I20260812 02:41:14.089708 29041 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502156971023-7698-0/minicluster-data/master-0/wal/instance:
uuid: "23dea4a07da149b5b72720ae55fe1981"
format_stamp: "Formatted at 2026-08-12 02:41:14 on dist-test-slave-mn4r"
server_key: "2c97561b4421f121c4f015f0c7e148b9"
server_key_iv: "087900426d2b64426c1ad97962ce288f"
server_key_version: "encryptionkey@0"
I20260812 02:41:14.095353 29041 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260812 02:41:14.099001 29057 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:41:14.100425 29041 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 02:41:14.100620 29041 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502156971023-7698-0/minicluster-data/master-0/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502156971023-7698-0/minicluster-data/master-0/wal
uuid: "23dea4a07da149b5b72720ae55fe1981"
format_stamp: "Formatted at 2026-08-12 02:41:14 on dist-test-slave-mn4r"
server_key: "2c97561b4421f121c4f015f0c7e148b9"
server_key_iv: "087900426d2b64426c1ad97962ce288f"
server_key_version: "encryptionkey@0"
I20260812 02:41:14.100847 29041 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502156971023-7698-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502156971023-7698-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502156971023-7698-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:41:14.135733 29041 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:41:14.136684 29041 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:41:14.137027 29041 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:41:14.156554 29041 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.190:40969
I20260812 02:41:14.156575 29120 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.190:40969 every 8 connection(s)
I20260812 02:41:14.157927 29041 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
I20260812 02:41:14.161835 29121 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:41:14.166844 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 29041
I20260812 02:41:14.167014 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502156971023-7698-0/minicluster-data/master-0/wal/instance
I20260812 02:41:14.167464 7698 external_mini_cluster.cc:1428] Setting key 06bd7c316e0bdb0beeda3fdaedcb6293
I20260812 02:41:14.172712 29121 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23dea4a07da149b5b72720ae55fe1981: Bootstrap starting.
I20260812 02:41:14.176411 29121 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 23dea4a07da149b5b72720ae55fe1981: Neither blocks nor log segments found. Creating new log.
I20260812 02:41:14.177654 29121 log.cc:824] T 00000000000000000000000000000000 P 23dea4a07da149b5b72720ae55fe1981: Log is configured to *not* fsync() on all Append() calls
I20260812 02:41:14.181026 29121 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23dea4a07da149b5b72720ae55fe1981: No bootstrap required, opened a new log
I20260812 02:41:14.181876 29124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:14.169526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51732 (local address 127.7.132.190:40969)
0812 02:41:14.170706 (+ 1180us) server_negotiation.cc:206] Beginning negotiation
0812 02:41:14.170721 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 02:41:14.170766 (+ 45us) server_negotiation.cc:407] Connection header received
0812 02:41:14.172438 (+ 1672us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:41:14.172461 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:41:14.172953 (+ 492us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:41:14.173289 (+ 336us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:41:14.173928 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:14.175310 (+ 1382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:14.177371 (+ 2061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:14.178193 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:14.178413 (+ 220us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:14.179577 (+ 1164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:41:14.179585 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:41:14.179598 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:41:14.180128 (+ 530us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:41:14.180206 (+ 78us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:41:14.180232 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:41:14.180309 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 02:41:14.180580 (+ 271us) server_negotiation.cc:299] Negotiation successful
0812 02:41:14.180914 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":735,"thread_start_us":417,"threads_started":1}
I20260812 02:41:14.188922 29121 raft_consensus.cc:348] T 00000000000000000000000000000000 P 23dea4a07da149b5b72720ae55fe1981 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23dea4a07da149b5b72720ae55fe1981" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 40969 } }
I20260812 02:41:14.189332 29121 raft_consensus.cc:374] T 00000000000000000000000000000000 P 23dea4a07da149b5b72720ae55fe1981 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:41:14.189455 29121 raft_consensus.cc:729] T 00000000000000000000000000000000 P 23dea4a07da149b5b72720ae55fe1981 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23dea4a07da149b5b72720ae55fe1981, State: Initialized, Role: FOLLOWER
I20260812 02:41:14.190037 29121 consensus_queue.cc:260] T 00000000000000000000000000000000 P 23dea4a07da149b5b72720ae55fe1981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23dea4a07da149b5b72720ae55fe1981" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 40969 } }
I20260812 02:41:14.190342 29121 raft_consensus.cc:388] T 00000000000000000000000000000000 P 23dea4a07da149b5b72720ae55fe1981 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 02:41:14.190510 29121 raft_consensus.cc:482] T 00000000000000000000000000000000 P 23dea4a07da149b5b72720ae55fe1981 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 02:41:14.190687 29121 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 23dea4a07da149b5b72720ae55fe1981 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:41:14.192929 29121 raft_consensus.cc:504] T 00000000000000000000000000000000 P 23dea4a07da149b5b72720ae55fe1981 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23dea4a07da149b5b72720ae55fe1981" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 40969 } }
I20260812 02:41:14.193506 29121 leader_election.cc:302] T 00000000000000000000000000000000 P 23dea4a07da149b5b72720ae55fe1981 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 23dea4a07da149b5b72720ae55fe1981; no voters:
I20260812 02:41:14.194258 29121 leader_election.cc:288] T 00000000000000000000000000000000 P 23dea4a07da149b5b72720ae55fe1981 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 02:41:14.194408 29126 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 23dea4a07da149b5b72720ae55fe1981 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:41:14.195151 29126 raft_consensus.cc:686] T 00000000000000000000000000000000 P 23dea4a07da149b5b72720ae55fe1981 [term 1 LEADER]: Becoming Leader. State: Replica: 23dea4a07da149b5b72720ae55fe1981, State: Running, Role: LEADER
I20260812 02:41:14.195637 29126 consensus_queue.cc:237] T 00000000000000000000000000000000 P 23dea4a07da149b5b72720ae55fe1981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23dea4a07da149b5b72720ae55fe1981" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 40969 } }
I20260812 02:41:14.196382 29121 sys_catalog.cc:564] T 00000000000000000000000000000000 P 23dea4a07da149b5b72720ae55fe1981 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:41:14.199036 29128 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23dea4a07da149b5b72720ae55fe1981 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 23dea4a07da149b5b72720ae55fe1981. Latest consensus state: current_term: 1 leader_uuid: "23dea4a07da149b5b72720ae55fe1981" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23dea4a07da149b5b72720ae55fe1981" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 40969 } } }
I20260812 02:41:14.199419 29128 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23dea4a07da149b5b72720ae55fe1981 [sys.catalog]: This master's current role is: LEADER
I20260812 02:41:14.205247 29127 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23dea4a07da149b5b72720ae55fe1981 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "23dea4a07da149b5b72720ae55fe1981" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23dea4a07da149b5b72720ae55fe1981" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 40969 } } }
I20260812 02:41:14.205628 29127 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23dea4a07da149b5b72720ae55fe1981 [sys.catalog]: This master's current role is: LEADER
I20260812 02:41:14.209501 29132 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:41:14.215152 29132 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 02:41:14.223831 29132 catalog_manager.cc:1347] Generated new cluster ID: 256336250243470b8c73ef8d0f650957
I20260812 02:41:14.223944 29132 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 02:41:14.235709 29132 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 02:41:14.236914 29132 catalog_manager.cc:1504] Loading token signing keys...
I20260812 02:41:14.249341 29132 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 23dea4a07da149b5b72720ae55fe1981: Generated new TSK 0
I20260812 02:41:14.250281 29132 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 02:41:14.256199 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502156971023-7698-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502156971023-7698-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502156971023-7698-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502156971023-7698-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.129:0
--local_ip_for_outbound_sockets=127.7.132.129
--webserver_interface=127.7.132.129
--webserver_port=0
--tserver_master_addrs=127.7.132.190:40969
--builtin_ntp_servers=127.7.132.148:44733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 02:41:14.607601 29148 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:41:14.607878 29148 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:41:14.608044 29148 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:41:14.615610 29148 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:41:14.615707 29148 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:41:14.615909 29148 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.129
I20260812 02:41:14.625072 29148 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:44733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502156971023-7698-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502156971023-7698-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502156971023-7698-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.132.129
--webserver_port=0
--tserver_master_addrs=127.7.132.190:40969
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.129
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502156971023-7698-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:41:14.626453 29148 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:41:14.628342 29148 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:41:14.642596 29156 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:41:14.642570 29154 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:41:14.643428 29153 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:41:14.644279 29148 server_base.cc:1034] running on GCE node
I20260812 02:41:14.644924 29148 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:41:14.645941 29148 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:41:14.647171 29148 hybrid_clock.cc:648] HybridClock initialized: now 1786502474647119 us; error 48 us; skew 500 ppm
I20260812 02:41:14.647519 29148 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 02:41:14.649801 29148 webserver.cc:460] Webserver started at http://127.7.132.129:35511/ using document root <none> and password file <none>
I20260812 02:41:14.650583 29148 fs_manager.cc:359] Metadata directory not provided
I20260812 02:41:14.650701 29148 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:41:14.651036 29148 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:41:14.653965 29148 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502156971023-7698-0/minicluster-data/ts-0/data/instance:
uuid: "3495979a70704c0c9cd6714d8238647a"
format_stamp: "Formatted at 2026-08-12 02:41:14 on dist-test-slave-mn4r"
server_key: "8168db829b2db923da5ca9be8923f74b"
server_key_iv: "296431d1d1c78f17b9c1103ee18c352a"
server_key_version: "encryptionkey@0"
I20260812 02:41:14.654831 29148 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502156971023-7698-0/minicluster-data/ts-0/wal/instance:
uuid: "3495979a70704c0c9cd6714d8238647a"
format_stamp: "Formatted at 2026-08-12 02:41:14 on dist-test-slave-mn4r"
server_key: "8168db829b2db923da5ca9be8923f74b"
server_key_iv: "296431d1d1c78f17b9c1103ee18c352a"
server_key_version: "encryptionkey@0"
I20260812 02:41:14.659977 29148 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 02:41:14.663794 29163 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:41:14.665112 29148 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 02:41:14.665313 29148 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502156971023-7698-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502156971023-7698-0/minicluster-data/ts-0/wal
uuid: "3495979a70704c0c9cd6714d8238647a"
format_stamp: "Formatted at 2026-08-12 02:41:14 on dist-test-slave-mn4r"
server_key: "8168db829b2db923da5ca9be8923f74b"
server_key_iv: "296431d1d1c78f17b9c1103ee18c352a"
server_key_version: "encryptionkey@0"
I20260812 02:41:14.665546 29148 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502156971023-7698-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502156971023-7698-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502156971023-7698-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:41:14.684024 29148 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:41:14.684870 29148 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:41:14.685187 29148 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:41:14.686203 29148 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:41:14.687709 29148 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:41:14.687835 29148 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:41:14.687958 29148 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:41:14.688052 29148 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:41:14.721312 29148 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.129:42221
I20260812 02:41:14.721369 29297 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.129:42221 every 8 connection(s)
I20260812 02:41:14.722729 29148 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502156971023-7698-0/minicluster-data/ts-0/data/info.pb
I20260812 02:41:14.729027 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 29148
I20260812 02:41:14.729240 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502156971023-7698-0/minicluster-data/ts-0/wal/instance
I20260812 02:41:14.729849 7698 external_mini_cluster.cc:1428] Setting key ab42f1a8b1079309f0768394a309dd61
I20260812 02:41:14.733184 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502156971023-7698-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502156971023-7698-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502156971023-7698-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502156971023-7698-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.130:0
--local_ip_for_outbound_sockets=127.7.132.130
--webserver_interface=127.7.132.130
--webserver_port=0
--tserver_master_addrs=127.7.132.190:40969
--builtin_ntp_servers=127.7.132.148:44733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 02:41:14.737270 29300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:14.725366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:49507 (local address 127.7.132.190:40969)
0812 02:41:14.725813 (+ 447us) server_negotiation.cc:206] Beginning negotiation
0812 02:41:14.725818 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:41:14.727105 (+ 1287us) server_negotiation.cc:407] Connection header received
0812 02:41:14.728974 (+ 1869us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:41:14.728978 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:41:14.729046 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:41:14.729176 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:41:14.730725 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:14.731810 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:14.734313 (+ 2503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:14.734735 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:14.734834 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:14.736506 (+ 1672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:41:14.736511 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:41:14.736514 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:41:14.736532 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:41:14.736578 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:41:14.736591 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:41:14.736656 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 02:41:14.736923 (+ 267us) server_negotiation.cc:299] Negotiation successful
0812 02:41:14.737011 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":239,"threads_started":1}
I20260812 02:41:14.738797 29302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:14.726001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:40969 (local address 127.7.132.129:49507)
0812 02:41:14.726935 (+ 934us) negotiation.cc:107] Waiting for socket to connect
0812 02:41:14.726983 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 02:41:14.728665 (+ 1682us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:41:14.729435 (+ 770us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:41:14.729445 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:41:14.729568 (+ 123us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:41:14.730546 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:41:14.730567 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:41:14.731998 (+ 1431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:41:14.732004 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:41:14.734157 (+ 2153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:41:14.734178 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:41:14.734970 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:41:14.734979 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:41:14.735337 (+ 358us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:14.736118 (+ 781us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 02:41:14.736124 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:41:14.736180 (+ 56us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 02:41:14.736293 (+ 113us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:41:14.736686 (+ 393us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:41:14.736700 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:41:14.736711 (+ 11us) client_negotiation.cc:769] Sending connection context
0812 02:41:14.736893 (+ 182us) client_negotiation.cc:240] Negotiation successful
0812 02:41:14.737337 (+ 444us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":505,"thread_start_us":266,"threads_started":1}
I20260812 02:41:14.740339 29298 heartbeater.cc:347] Connected to a master server at 127.7.132.190:40969
I20260812 02:41:14.740665 29298 heartbeater.cc:464] Registering TS with master...
I20260812 02:41:14.741411 29298 heartbeater.cc:511] Master 127.7.132.190:40969 requested a full tablet report, sending...
I20260812 02:41:14.743552 29078 ts_manager.cc:194] Registered new tserver with Master: 3495979a70704c0c9cd6714d8238647a (127.7.132.129:42221)
I20260812 02:41:14.745095 29078 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.132.129:49507
W20260812 02:41:15.077157 29303 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:41:15.077514 29303 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:41:15.077685 29303 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:41:15.085302 29303 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:41:15.085455 29303 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:41:15.085644 29303 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.130
I20260812 02:41:15.095211 29303 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:44733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502156971023-7698-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502156971023-7698-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502156971023-7698-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.132.130
--webserver_port=0
--tserver_master_addrs=127.7.132.190:40969
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.130
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502156971023-7698-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:41:15.096690 29303 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:41:15.098667 29303 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:41:15.112625 29311 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:41:15.112732 29313 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:41:15.113118 29310 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:41:15.113605 29303 server_base.cc:1034] running on GCE node
I20260812 02:41:15.114319 29303 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:41:15.115271 29303 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:41:15.116489 29303 hybrid_clock.cc:648] HybridClock initialized: now 1786502475116442 us; error 41 us; skew 500 ppm
I20260812 02:41:15.116832 29303 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 02:41:15.119202 29303 webserver.cc:460] Webserver started at http://127.7.132.130:34331/ using document root <none> and password file <none>
I20260812 02:41:15.119917 29303 fs_manager.cc:359] Metadata directory not provided
I20260812 02:41:15.120047 29303 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:41:15.120393 29303 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:41:15.123402 29303 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502156971023-7698-0/minicluster-data/ts-1/data/instance:
uuid: "f4039ccfc39244e9821ed45c2ecdae76"
format_stamp: "Formatted at 2026-08-12 02:41:15 on dist-test-slave-mn4r"
server_key: "ddd8ba3ed17c38bc408c033529967d21"
server_key_iv: "3528fbd631e3d9b4dfcf984606ab73bc"
server_key_version: "encryptionkey@0"
I20260812 02:41:15.124245 29303 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502156971023-7698-0/minicluster-data/ts-1/wal/instance:
uuid: "f4039ccfc39244e9821ed45c2ecdae76"
format_stamp: "Formatted at 2026-08-12 02:41:15 on dist-test-slave-mn4r"
server_key: "ddd8ba3ed17c38bc408c033529967d21"
server_key_iv: "3528fbd631e3d9b4dfcf984606ab73bc"
server_key_version: "encryptionkey@0"
I20260812 02:41:15.129509 29303 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260812 02:41:15.133172 29320 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:41:15.134764 29303 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 02:41:15.134953 29303 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502156971023-7698-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502156971023-7698-0/minicluster-data/ts-1/wal
uuid: "f4039ccfc39244e9821ed45c2ecdae76"
format_stamp: "Formatted at 2026-08-12 02:41:15 on dist-test-slave-mn4r"
server_key: "ddd8ba3ed17c38bc408c033529967d21"
server_key_iv: "3528fbd631e3d9b4dfcf984606ab73bc"
server_key_version: "encryptionkey@0"
I20260812 02:41:15.135182 29303 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502156971023-7698-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502156971023-7698-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502156971023-7698-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:41:15.159186 29303 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:41:15.160072 29303 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:41:15.160394 29303 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:41:15.161494 29303 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:41:15.163288 29303 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:41:15.163411 29303 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:41:15.163545 29303 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:41:15.163632 29303 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:41:15.204886 29303 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.130:33769
I20260812 02:41:15.205009 29452 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.130:33769 every 8 connection(s)
I20260812 02:41:15.206326 29303 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502156971023-7698-0/minicluster-data/ts-1/data/info.pb
I20260812 02:41:15.216454 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 29303
I20260812 02:41:15.216670 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502156971023-7698-0/minicluster-data/ts-1/wal/instance
I20260812 02:41:15.217249 7698 external_mini_cluster.cc:1428] Setting key f7f29014fb5612966aa6291f03bc570b
I20260812 02:41:15.219503 29300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:15.209064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:39461 (local address 127.7.132.190:40969)
0812 02:41:15.209256 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0812 02:41:15.209260 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 02:41:15.210900 (+ 1640us) server_negotiation.cc:407] Connection header received
0812 02:41:15.212671 (+ 1771us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:41:15.212675 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:41:15.212746 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:41:15.212839 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:41:15.213877 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:15.214712 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:15.216388 (+ 1676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:15.216897 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:15.216999 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:15.218682 (+ 1683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:41:15.218687 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:41:15.218690 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:41:15.218707 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:41:15.218739 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:41:15.218751 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:41:15.218823 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 02:41:15.219089 (+ 266us) server_negotiation.cc:299] Negotiation successful
0812 02:41:15.219173 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260812 02:41:15.220546 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502156971023-7698-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502156971023-7698-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502156971023-7698-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502156971023-7698-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.131:0
--local_ip_for_outbound_sockets=127.7.132.131
--webserver_interface=127.7.132.131
--webserver_port=0
--tserver_master_addrs=127.7.132.190:40969
--builtin_ntp_servers=127.7.132.148:44733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 02:41:15.220650 29455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:15.209687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:40969 (local address 127.7.132.130:39461)
0812 02:41:15.210727 (+ 1040us) negotiation.cc:107] Waiting for socket to connect
0812 02:41:15.210771 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 02:41:15.212406 (+ 1635us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:41:15.213040 (+ 634us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:41:15.213048 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:41:15.213148 (+ 100us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:41:15.213728 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:41:15.213745 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:41:15.214844 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:41:15.214847 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:41:15.216234 (+ 1387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:41:15.216254 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:41:15.217145 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:41:15.217153 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:41:15.217482 (+ 329us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:15.218336 (+ 854us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 02:41:15.218338 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:41:15.218378 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 02:41:15.218508 (+ 130us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:41:15.218855 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:41:15.218866 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:41:15.218874 (+ 8us) client_negotiation.cc:769] Sending connection context
0812 02:41:15.219065 (+ 191us) client_negotiation.cc:240] Negotiation successful
0812 02:41:15.219512 (+ 447us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":600,"thread_start_us":411,"threads_started":1}
I20260812 02:41:15.222401 29453 heartbeater.cc:347] Connected to a master server at 127.7.132.190:40969
I20260812 02:41:15.222743 29453 heartbeater.cc:464] Registering TS with master...
I20260812 02:41:15.223488 29453 heartbeater.cc:511] Master 127.7.132.190:40969 requested a full tablet report, sending...
I20260812 02:41:15.224964 29078 ts_manager.cc:194] Registered new tserver with Master: f4039ccfc39244e9821ed45c2ecdae76 (127.7.132.130:33769)
I20260812 02:41:15.225823 29078 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.132.130:39461
W20260812 02:41:15.577643 29456 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:41:15.577936 29456 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:41:15.578025 29456 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:41:15.585281 29456 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:41:15.585389 29456 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:41:15.585496 29456 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.131
I20260812 02:41:15.594877 29456 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:44733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502156971023-7698-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502156971023-7698-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502156971023-7698-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.132.131
--webserver_port=0
--tserver_master_addrs=127.7.132.190:40969
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.131
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502156971023-7698-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:41:15.596256 29456 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:41:15.598182 29456 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:41:15.611764 29465 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:41:15.611744 29463 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:41:15.611861 29462 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:41:15.612854 29456 server_base.cc:1034] running on GCE node
I20260812 02:41:15.613508 29456 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:41:15.614579 29456 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:41:15.615813 29456 hybrid_clock.cc:648] HybridClock initialized: now 1786502475615769 us; error 41 us; skew 500 ppm
I20260812 02:41:15.616154 29456 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 02:41:15.618433 29456 webserver.cc:460] Webserver started at http://127.7.132.131:44689/ using document root <none> and password file <none>
I20260812 02:41:15.619153 29456 fs_manager.cc:359] Metadata directory not provided
I20260812 02:41:15.619293 29456 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:41:15.619632 29456 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:41:15.622627 29456 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502156971023-7698-0/minicluster-data/ts-2/data/instance:
uuid: "7b12370667f144f5aa0495277e20e616"
format_stamp: "Formatted at 2026-08-12 02:41:15 on dist-test-slave-mn4r"
server_key: "bf59948896cfa282ed505f4a0f468ac3"
server_key_iv: "49def92eb8b36fba58bf48852c4067e6"
server_key_version: "encryptionkey@0"
I20260812 02:41:15.623483 29456 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502156971023-7698-0/minicluster-data/ts-2/wal/instance:
uuid: "7b12370667f144f5aa0495277e20e616"
format_stamp: "Formatted at 2026-08-12 02:41:15 on dist-test-slave-mn4r"
server_key: "bf59948896cfa282ed505f4a0f468ac3"
server_key_iv: "49def92eb8b36fba58bf48852c4067e6"
server_key_version: "encryptionkey@0"
I20260812 02:41:15.629066 29456 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260812 02:41:15.632761 29472 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:41:15.634322 29456 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 02:41:15.634516 29456 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502156971023-7698-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502156971023-7698-0/minicluster-data/ts-2/wal
uuid: "7b12370667f144f5aa0495277e20e616"
format_stamp: "Formatted at 2026-08-12 02:41:15 on dist-test-slave-mn4r"
server_key: "bf59948896cfa282ed505f4a0f468ac3"
server_key_iv: "49def92eb8b36fba58bf48852c4067e6"
server_key_version: "encryptionkey@0"
I20260812 02:41:15.634743 29456 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502156971023-7698-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502156971023-7698-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502156971023-7698-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:41:15.654920 29456 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:41:15.655790 29456 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:41:15.656116 29456 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:41:15.657140 29456 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:41:15.658675 29456 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:41:15.658800 29456 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:41:15.658936 29456 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:41:15.659032 29456 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:41:15.693897 29456 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.131:39705
I20260812 02:41:15.694020 29601 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.131:39705 every 8 connection(s)
I20260812 02:41:15.695318 29456 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502156971023-7698-0/minicluster-data/ts-2/data/info.pb
I20260812 02:41:15.703357 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 29456
I20260812 02:41:15.703554 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502156971023-7698-0/minicluster-data/ts-2/wal/instance
I20260812 02:41:15.704021 7698 external_mini_cluster.cc:1428] Setting key 9573bea2bce588a8c77a7560256ca0e9
I20260812 02:41:15.709025 29300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:15.698179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:54387 (local address 127.7.132.190:40969)
0812 02:41:15.698360 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0812 02:41:15.698365 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:41:15.699879 (+ 1514us) server_negotiation.cc:407] Connection header received
0812 02:41:15.701758 (+ 1879us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:41:15.701765 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:41:15.701866 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:41:15.701994 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:41:15.703217 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:15.704155 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:15.706013 (+ 1858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:15.706569 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:15.706706 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:15.708181 (+ 1475us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:41:15.708186 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:41:15.708189 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:41:15.708207 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:41:15.708239 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:41:15.708251 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:41:15.708355 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0812 02:41:15.708593 (+ 238us) server_negotiation.cc:299] Negotiation successful
0812 02:41:15.708697 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 02:41:15.710183 29604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:15.698764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:40969 (local address 127.7.132.131:54387)
0812 02:41:15.699710 (+ 946us) negotiation.cc:107] Waiting for socket to connect
0812 02:41:15.699753 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 02:41:15.701478 (+ 1725us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:41:15.702261 (+ 783us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:41:15.702269 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:41:15.702358 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:41:15.703000 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:41:15.703038 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:41:15.704335 (+ 1297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:41:15.704341 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:41:15.705862 (+ 1521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:41:15.705874 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:41:15.706810 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:41:15.706818 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:41:15.707060 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:15.707781 (+ 721us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 02:41:15.707783 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:41:15.707823 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 02:41:15.707905 (+ 82us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:41:15.708355 (+ 450us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:41:15.708366 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:41:15.708376 (+ 10us) client_negotiation.cc:769] Sending connection context
0812 02:41:15.708583 (+ 207us) client_negotiation.cc:240] Negotiation successful
0812 02:41:15.708994 (+ 411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":510,"thread_start_us":315,"threads_started":1}
I20260812 02:41:15.711493 29602 heartbeater.cc:347] Connected to a master server at 127.7.132.190:40969
I20260812 02:41:15.711803 29602 heartbeater.cc:464] Registering TS with master...
I20260812 02:41:15.712540 29602 heartbeater.cc:511] Master 127.7.132.190:40969 requested a full tablet report, sending...
I20260812 02:41:15.713899 29078 ts_manager.cc:194] Registered new tserver with Master: 7b12370667f144f5aa0495277e20e616 (127.7.132.131:39705)
I20260812 02:41:15.714752 29078 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.132.131:54387
I20260812 02:41:15.719303 7698 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 02:41:15.729851 29300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:15.723656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51736 (local address 127.7.132.190:40969)
0812 02:41:15.723884 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0812 02:41:15.723893 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 02:41:15.724342 (+ 449us) server_negotiation.cc:407] Connection header received
0812 02:41:15.724543 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:41:15.724547 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:41:15.724616 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:41:15.724696 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:41:15.725275 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:15.725963 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:15.727625 (+ 1662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:15.728038 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:15.728136 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:15.729196 (+ 1060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:41:15.729200 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:41:15.729203 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:41:15.729220 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:41:15.729256 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:41:15.729267 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:41:15.729335 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 02:41:15.729455 (+ 120us) server_negotiation.cc:299] Negotiation successful
0812 02:41:15.729535 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260812 02:41:15.733546 29078 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51736:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 02:41:15.741179 29078 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 02:41:15.748250 29298 heartbeater.cc:503] Master 127.7.132.190:40969 was elected leader, sending a full tablet report...
I20260812 02:41:15.776811 29617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:15.754710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56734 (local address 127.7.132.129:42221)
0812 02:41:15.765496 (+ 10786us) server_negotiation.cc:206] Beginning negotiation
0812 02:41:15.765509 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 02:41:15.765545 (+ 36us) server_negotiation.cc:407] Connection header received
0812 02:41:15.765660 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:41:15.765672 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:41:15.765986 (+ 314us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:41:15.766220 (+ 234us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:41:15.768939 (+ 2719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:15.770619 (+ 1680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:15.773931 (+ 3312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:15.775525 (+ 1594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:15.775672 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:15.775971 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0812 02:41:15.776254 (+ 283us) server_negotiation.cc:299] Negotiation successful
0812 02:41:15.776434 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10557,"thread_start_us":10397,"threads_started":1}
I20260812 02:41:15.776855 29615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:15.754147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:42221 (local address 127.0.0.1:56734)
0812 02:41:15.764531 (+ 10384us) negotiation.cc:107] Waiting for socket to connect
0812 02:41:15.764593 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0812 02:41:15.764797 (+ 204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:41:15.767923 (+ 3126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:41:15.767936 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:41:15.767977 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:41:15.768702 (+ 725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:41:15.768725 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:41:15.770785 (+ 2060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:41:15.770792 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:41:15.773699 (+ 2907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:41:15.773716 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:41:15.775742 (+ 2026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:41:15.775749 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:41:15.775982 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:15.776040 (+ 58us) client_negotiation.cc:769] Sending connection context
0812 02:41:15.776192 (+ 152us) client_negotiation.cc:240] Negotiation successful
0812 02:41:15.776399 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10056,"mutex_wait_us":52,"thread_start_us":15301,"threads_started":1}
I20260812 02:41:15.777842 29614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:15.755193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.130:33769 (local address 127.0.0.1:33648)
0812 02:41:15.769253 (+ 14060us) negotiation.cc:107] Waiting for socket to connect
0812 02:41:15.769268 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 02:41:15.769425 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:41:15.770330 (+ 905us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:41:15.770335 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:41:15.770361 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:41:15.770728 (+ 367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:41:15.770742 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:41:15.772066 (+ 1324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:41:15.772071 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:41:15.774864 (+ 2793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:41:15.774881 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:41:15.777053 (+ 2172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:41:15.777063 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:41:15.777230 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:15.777265 (+ 35us) client_negotiation.cc:769] Sending connection context
0812 02:41:15.777357 (+ 92us) client_negotiation.cc:240] Negotiation successful
0812 02:41:15.777511 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13963,"mutex_wait_us":118,"thread_start_us":11910,"threads_started":1}
I20260812 02:41:15.778119 29618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:15.754284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.131:39705 (local address 127.0.0.1:50344)
0812 02:41:15.767497 (+ 13213us) negotiation.cc:107] Waiting for socket to connect
0812 02:41:15.767523 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0812 02:41:15.767691 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:41:15.769820 (+ 2129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:41:15.769825 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:41:15.769852 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:41:15.770352 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:41:15.770370 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:41:15.772594 (+ 2224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:41:15.772600 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:41:15.774866 (+ 2266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:41:15.774879 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:41:15.777089 (+ 2210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:41:15.777096 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:41:15.777316 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:15.777358 (+ 42us) client_negotiation.cc:769] Sending connection context
0812 02:41:15.777487 (+ 129us) client_negotiation.cc:240] Negotiation successful
0812 02:41:15.777627 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13030,"mutex_wait_us":31,"thread_start_us":9766,"threads_started":1}
I20260812 02:41:15.778169 29616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:15.754318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50344 (local address 127.7.132.131:39705)
0812 02:41:15.769185 (+ 14867us) server_negotiation.cc:206] Beginning negotiation
0812 02:41:15.769196 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 02:41:15.769235 (+ 39us) server_negotiation.cc:407] Connection header received
0812 02:41:15.769334 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:41:15.769342 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:41:15.769619 (+ 277us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:41:15.769816 (+ 197us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:41:15.770637 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:15.772398 (+ 1761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:15.775093 (+ 2695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:15.776446 (+ 1353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:15.776598 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:15.776920 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0812 02:41:15.777549 (+ 629us) server_negotiation.cc:299] Negotiation successful
0812 02:41:15.777724 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14687,"thread_start_us":14522,"threads_started":1}
I20260812 02:41:15.778725 29619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:15.755890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33648 (local address 127.7.132.130:33769)
0812 02:41:15.766861 (+ 10971us) server_negotiation.cc:206] Beginning negotiation
0812 02:41:15.766875 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 02:41:15.769393 (+ 2518us) server_negotiation.cc:407] Connection header received
0812 02:41:15.769790 (+ 397us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:41:15.769799 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:41:15.770072 (+ 273us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:41:15.770329 (+ 257us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:41:15.770930 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:15.771908 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:15.775135 (+ 3227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:15.776498 (+ 1363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:15.776638 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:15.776886 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0812 02:41:15.778184 (+ 1298us) server_negotiation.cc:299] Negotiation successful
0812 02:41:15.778359 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10724,"thread_start_us":10567,"threads_started":1}
I20260812 02:41:15.781531 29220 tablet_service.cc:1467] Processing CreateTablet for tablet 2a291bfa1f654e029f35a587d5b77131 (DEFAULT_TABLE table=test-table [id=3a21022f6d0f4aeb9fc2a5b02df23ebe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:41:15.781733 29377 tablet_service.cc:1467] Processing CreateTablet for tablet 2a291bfa1f654e029f35a587d5b77131 (DEFAULT_TABLE table=test-table [id=3a21022f6d0f4aeb9fc2a5b02df23ebe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:41:15.781818 29525 tablet_service.cc:1467] Processing CreateTablet for tablet 2a291bfa1f654e029f35a587d5b77131 (DEFAULT_TABLE table=test-table [id=3a21022f6d0f4aeb9fc2a5b02df23ebe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:41:15.783193 29220 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a291bfa1f654e029f35a587d5b77131. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:41:15.783214 29525 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a291bfa1f654e029f35a587d5b77131. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:41:15.783237 29377 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a291bfa1f654e029f35a587d5b77131. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:41:15.795792 29621 tablet_bootstrap.cc:492] T 2a291bfa1f654e029f35a587d5b77131 P f4039ccfc39244e9821ed45c2ecdae76: Bootstrap starting.
I20260812 02:41:15.798214 29622 tablet_bootstrap.cc:492] T 2a291bfa1f654e029f35a587d5b77131 P 3495979a70704c0c9cd6714d8238647a: Bootstrap starting.
I20260812 02:41:15.800855 29621 tablet_bootstrap.cc:654] T 2a291bfa1f654e029f35a587d5b77131 P f4039ccfc39244e9821ed45c2ecdae76: Neither blocks nor log segments found. Creating new log.
I20260812 02:41:15.802364 29622 tablet_bootstrap.cc:654] T 2a291bfa1f654e029f35a587d5b77131 P 3495979a70704c0c9cd6714d8238647a: Neither blocks nor log segments found. Creating new log.
I20260812 02:41:15.802541 29621 log.cc:824] T 2a291bfa1f654e029f35a587d5b77131 P f4039ccfc39244e9821ed45c2ecdae76: Log is configured to *not* fsync() on all Append() calls
I20260812 02:41:15.802868 29623 tablet_bootstrap.cc:492] T 2a291bfa1f654e029f35a587d5b77131 P 7b12370667f144f5aa0495277e20e616: Bootstrap starting.
I20260812 02:41:15.803905 29622 log.cc:824] T 2a291bfa1f654e029f35a587d5b77131 P 3495979a70704c0c9cd6714d8238647a: Log is configured to *not* fsync() on all Append() calls
I20260812 02:41:15.806454 29623 tablet_bootstrap.cc:654] T 2a291bfa1f654e029f35a587d5b77131 P 7b12370667f144f5aa0495277e20e616: Neither blocks nor log segments found. Creating new log.
I20260812 02:41:15.807556 29623 log.cc:824] T 2a291bfa1f654e029f35a587d5b77131 P 7b12370667f144f5aa0495277e20e616: Log is configured to *not* fsync() on all Append() calls
I20260812 02:41:15.808705 29622 tablet_bootstrap.cc:492] T 2a291bfa1f654e029f35a587d5b77131 P 3495979a70704c0c9cd6714d8238647a: No bootstrap required, opened a new log
I20260812 02:41:15.809051 29622 ts_tablet_manager.cc:1397] T 2a291bfa1f654e029f35a587d5b77131 P 3495979a70704c0c9cd6714d8238647a: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.004s
I20260812 02:41:15.809128 29621 tablet_bootstrap.cc:492] T 2a291bfa1f654e029f35a587d5b77131 P f4039ccfc39244e9821ed45c2ecdae76: No bootstrap required, opened a new log
I20260812 02:41:15.809399 29621 ts_tablet_manager.cc:1397] T 2a291bfa1f654e029f35a587d5b77131 P f4039ccfc39244e9821ed45c2ecdae76: Time spent bootstrapping tablet: real 0.014s user 0.007s sys 0.004s
I20260812 02:41:15.812359 29623 tablet_bootstrap.cc:492] T 2a291bfa1f654e029f35a587d5b77131 P 7b12370667f144f5aa0495277e20e616: No bootstrap required, opened a new log
I20260812 02:41:15.812619 29623 ts_tablet_manager.cc:1397] T 2a291bfa1f654e029f35a587d5b77131 P 7b12370667f144f5aa0495277e20e616: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.005s
I20260812 02:41:15.817065 29622 raft_consensus.cc:348] T 2a291bfa1f654e029f35a587d5b77131 P 3495979a70704c0c9cd6714d8238647a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b12370667f144f5aa0495277e20e616" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 39705 } } peers { permanent_uuid: "3495979a70704c0c9cd6714d8238647a" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 42221 } } peers { permanent_uuid: "f4039ccfc39244e9821ed45c2ecdae76" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 33769 } }
I20260812 02:41:15.817471 29622 raft_consensus.cc:374] T 2a291bfa1f654e029f35a587d5b77131 P 3495979a70704c0c9cd6714d8238647a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:41:15.817628 29622 raft_consensus.cc:729] T 2a291bfa1f654e029f35a587d5b77131 P 3495979a70704c0c9cd6714d8238647a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3495979a70704c0c9cd6714d8238647a, State: Initialized, Role: FOLLOWER
I20260812 02:41:15.818346 29622 consensus_queue.cc:260] T 2a291bfa1f654e029f35a587d5b77131 P 3495979a70704c0c9cd6714d8238647a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b12370667f144f5aa0495277e20e616" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 39705 } } peers { permanent_uuid: "3495979a70704c0c9cd6714d8238647a" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 42221 } } peers { permanent_uuid: "f4039ccfc39244e9821ed45c2ecdae76" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 33769 } }
I20260812 02:41:15.819994 29621 raft_consensus.cc:348] T 2a291bfa1f654e029f35a587d5b77131 P f4039ccfc39244e9821ed45c2ecdae76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b12370667f144f5aa0495277e20e616" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 39705 } } peers { permanent_uuid: "3495979a70704c0c9cd6714d8238647a" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 42221 } } peers { permanent_uuid: "f4039ccfc39244e9821ed45c2ecdae76" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 33769 } }
I20260812 02:41:15.820323 29622 ts_tablet_manager.cc:1428] T 2a291bfa1f654e029f35a587d5b77131 P 3495979a70704c0c9cd6714d8238647a: Time spent starting tablet: real 0.011s user 0.007s sys 0.003s
I20260812 02:41:15.820442 29621 raft_consensus.cc:374] T 2a291bfa1f654e029f35a587d5b77131 P f4039ccfc39244e9821ed45c2ecdae76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:41:15.820574 29621 raft_consensus.cc:729] T 2a291bfa1f654e029f35a587d5b77131 P f4039ccfc39244e9821ed45c2ecdae76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4039ccfc39244e9821ed45c2ecdae76, State: Initialized, Role: FOLLOWER
I20260812 02:41:15.821180 29621 consensus_queue.cc:260] T 2a291bfa1f654e029f35a587d5b77131 P f4039ccfc39244e9821ed45c2ecdae76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b12370667f144f5aa0495277e20e616" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 39705 } } peers { permanent_uuid: "3495979a70704c0c9cd6714d8238647a" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 42221 } } peers { permanent_uuid: "f4039ccfc39244e9821ed45c2ecdae76" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 33769 } }
I20260812 02:41:15.822429 29453 heartbeater.cc:503] Master 127.7.132.190:40969 was elected leader, sending a full tablet report...
I20260812 02:41:15.823030 29621 ts_tablet_manager.cc:1428] T 2a291bfa1f654e029f35a587d5b77131 P f4039ccfc39244e9821ed45c2ecdae76: Time spent starting tablet: real 0.013s user 0.014s sys 0.001s
I20260812 02:41:15.825003 29623 raft_consensus.cc:348] T 2a291bfa1f654e029f35a587d5b77131 P 7b12370667f144f5aa0495277e20e616 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b12370667f144f5aa0495277e20e616" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 39705 } } peers { permanent_uuid: "3495979a70704c0c9cd6714d8238647a" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 42221 } } peers { permanent_uuid: "f4039ccfc39244e9821ed45c2ecdae76" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 33769 } }
I20260812 02:41:15.826097 29623 raft_consensus.cc:374] T 2a291bfa1f654e029f35a587d5b77131 P 7b12370667f144f5aa0495277e20e616 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:41:15.826284 29623 raft_consensus.cc:729] T 2a291bfa1f654e029f35a587d5b77131 P 7b12370667f144f5aa0495277e20e616 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b12370667f144f5aa0495277e20e616, State: Initialized, Role: FOLLOWER
I20260812 02:41:15.826876 29623 consensus_queue.cc:260] T 2a291bfa1f654e029f35a587d5b77131 P 7b12370667f144f5aa0495277e20e616 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b12370667f144f5aa0495277e20e616" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 39705 } } peers { permanent_uuid: "3495979a70704c0c9cd6714d8238647a" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 42221 } } peers { permanent_uuid: "f4039ccfc39244e9821ed45c2ecdae76" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 33769 } }
I20260812 02:41:15.828001 29602 heartbeater.cc:503] Master 127.7.132.190:40969 was elected leader, sending a full tablet report...
I20260812 02:41:15.828337 29623 ts_tablet_manager.cc:1428] T 2a291bfa1f654e029f35a587d5b77131 P 7b12370667f144f5aa0495277e20e616: Time spent starting tablet: real 0.016s user 0.012s sys 0.002s
I20260812 02:41:15.912829 29629 raft_consensus.cc:482] T 2a291bfa1f654e029f35a587d5b77131 P f4039ccfc39244e9821ed45c2ecdae76 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 02:41:15.913146 29629 raft_consensus.cc:504] T 2a291bfa1f654e029f35a587d5b77131 P f4039ccfc39244e9821ed45c2ecdae76 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b12370667f144f5aa0495277e20e616" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 39705 } } peers { permanent_uuid: "3495979a70704c0c9cd6714d8238647a" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 42221 } } peers { permanent_uuid: "f4039ccfc39244e9821ed45c2ecdae76" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 33769 } }
I20260812 02:41:15.915062 29629 leader_election.cc:288] T 2a291bfa1f654e029f35a587d5b77131 P f4039ccfc39244e9821ed45c2ecdae76 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7b12370667f144f5aa0495277e20e616 (127.7.132.131:39705), 3495979a70704c0c9cd6714d8238647a (127.7.132.129:42221)
I20260812 02:41:15.922155 29616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:15.916306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:52395 (local address 127.7.132.131:39705)
0812 02:41:15.916549 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0812 02:41:15.916558 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 02:41:15.916733 (+ 175us) server_negotiation.cc:407] Connection header received
0812 02:41:15.916936 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:41:15.916940 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:41:15.917005 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:41:15.917116 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:41:15.917676 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:15.918474 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:15.920353 (+ 1879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:15.921225 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:15.921403 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:15.921472 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 02:41:15.921761 (+ 289us) server_negotiation.cc:299] Negotiation successful
0812 02:41:15.921880 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260812 02:41:15.922283 29634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:15.915507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.131:39705 (local address 127.7.132.130:52395)
0812 02:41:15.916599 (+ 1092us) negotiation.cc:107] Waiting for socket to connect
0812 02:41:15.916623 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0812 02:41:15.916792 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:41:15.917134 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:41:15.917139 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:41:15.917176 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:41:15.917515 (+ 339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:41:15.917524 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:41:15.918601 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:41:15.918609 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:41:15.920134 (+ 1525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:41:15.920147 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:41:15.921412 (+ 1265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:41:15.921418 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:41:15.921580 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:15.921605 (+ 25us) client_negotiation.cc:769] Sending connection context
0812 02:41:15.921720 (+ 115us) client_negotiation.cc:240] Negotiation successful
0812 02:41:15.921843 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":889,"thread_start_us":1049,"threads_started":1}
I20260812 02:41:15.922500 29632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:15.915560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:42221 (local address 127.7.132.130:38803)
0812 02:41:15.916905 (+ 1345us) negotiation.cc:107] Waiting for socket to connect
0812 02:41:15.916927 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 02:41:15.917068 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:41:15.917415 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:41:15.917419 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:41:15.917448 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:41:15.917802 (+ 354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:41:15.917810 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:41:15.918846 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:41:15.918851 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:41:15.920308 (+ 1457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:41:15.920320 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:41:15.921665 (+ 1345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:41:15.921672 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:41:15.921826 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:15.921849 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 02:41:15.921966 (+ 117us) client_negotiation.cc:240] Negotiation successful
0812 02:41:15.922166 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1210,"thread_start_us":621,"threads_started":1}
I20260812 02:41:15.922699 29617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:15.915726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:38803 (local address 127.7.132.129:42221)
0812 02:41:15.915986 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0812 02:41:15.915996 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 02:41:15.917020 (+ 1024us) server_negotiation.cc:407] Connection header received
0812 02:41:15.917214 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:41:15.917219 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:41:15.917290 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:41:15.917392 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:41:15.917964 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:15.918699 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:15.920481 (+ 1782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:15.921478 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:15.921658 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:15.921724 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 02:41:15.921992 (+ 268us) server_negotiation.cc:299] Negotiation successful
0812 02:41:15.922153 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260812 02:41:15.923586 29549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a291bfa1f654e029f35a587d5b77131" candidate_uuid: "f4039ccfc39244e9821ed45c2ecdae76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b12370667f144f5aa0495277e20e616" is_pre_election: true
I20260812 02:41:15.923586 29245 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a291bfa1f654e029f35a587d5b77131" candidate_uuid: "f4039ccfc39244e9821ed45c2ecdae76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3495979a70704c0c9cd6714d8238647a" is_pre_election: true
I20260812 02:41:15.924202 29549 raft_consensus.cc:2454] T 2a291bfa1f654e029f35a587d5b77131 P 7b12370667f144f5aa0495277e20e616 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f4039ccfc39244e9821ed45c2ecdae76 in term 0.
I20260812 02:41:15.924202 29245 raft_consensus.cc:2454] T 2a291bfa1f654e029f35a587d5b77131 P 3495979a70704c0c9cd6714d8238647a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f4039ccfc39244e9821ed45c2ecdae76 in term 0.
I20260812 02:41:15.925105 29324 leader_election.cc:302] T 2a291bfa1f654e029f35a587d5b77131 P f4039ccfc39244e9821ed45c2ecdae76 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 3495979a70704c0c9cd6714d8238647a, 7b12370667f144f5aa0495277e20e616, f4039ccfc39244e9821ed45c2ecdae76; no voters:
I20260812 02:41:15.925603 29629 raft_consensus.cc:2781] T 2a291bfa1f654e029f35a587d5b77131 P f4039ccfc39244e9821ed45c2ecdae76 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 02:41:15.925792 29629 raft_consensus.cc:482] T 2a291bfa1f654e029f35a587d5b77131 P f4039ccfc39244e9821ed45c2ecdae76 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 02:41:15.925921 29629 raft_consensus.cc:3037] T 2a291bfa1f654e029f35a587d5b77131 P f4039ccfc39244e9821ed45c2ecdae76 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:41:15.928464 29629 raft_consensus.cc:504] T 2a291bfa1f654e029f35a587d5b77131 P f4039ccfc39244e9821ed45c2ecdae76 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b12370667f144f5aa0495277e20e616" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 39705 } } peers { permanent_uuid: "3495979a70704c0c9cd6714d8238647a" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 42221 } } peers { permanent_uuid: "f4039ccfc39244e9821ed45c2ecdae76" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 33769 } }
I20260812 02:41:15.929100 29629 leader_election.cc:288] T 2a291bfa1f654e029f35a587d5b77131 P f4039ccfc39244e9821ed45c2ecdae76 [CANDIDATE]: Term 1 election: Requested vote from peers 7b12370667f144f5aa0495277e20e616 (127.7.132.131:39705), 3495979a70704c0c9cd6714d8238647a (127.7.132.129:42221)
I20260812 02:41:15.929646 29245 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a291bfa1f654e029f35a587d5b77131" candidate_uuid: "f4039ccfc39244e9821ed45c2ecdae76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3495979a70704c0c9cd6714d8238647a"
I20260812 02:41:15.929667 29549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a291bfa1f654e029f35a587d5b77131" candidate_uuid: "f4039ccfc39244e9821ed45c2ecdae76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b12370667f144f5aa0495277e20e616"
I20260812 02:41:15.929854 29549 raft_consensus.cc:3037] T 2a291bfa1f654e029f35a587d5b77131 P 7b12370667f144f5aa0495277e20e616 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:41:15.929883 29245 raft_consensus.cc:3037] T 2a291bfa1f654e029f35a587d5b77131 P 3495979a70704c0c9cd6714d8238647a [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:41:15.932116 29549 raft_consensus.cc:2454] T 2a291bfa1f654e029f35a587d5b77131 P 7b12370667f144f5aa0495277e20e616 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f4039ccfc39244e9821ed45c2ecdae76 in term 1.
I20260812 02:41:15.932415 29245 raft_consensus.cc:2454] T 2a291bfa1f654e029f35a587d5b77131 P 3495979a70704c0c9cd6714d8238647a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f4039ccfc39244e9821ed45c2ecdae76 in term 1.
I20260812 02:41:15.932511 29324 leader_election.cc:302] T 2a291bfa1f654e029f35a587d5b77131 P f4039ccfc39244e9821ed45c2ecdae76 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7b12370667f144f5aa0495277e20e616, f4039ccfc39244e9821ed45c2ecdae76; no voters:
I20260812 02:41:15.932863 29629 raft_consensus.cc:2781] T 2a291bfa1f654e029f35a587d5b77131 P f4039ccfc39244e9821ed45c2ecdae76 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:41:15.933408 29629 raft_consensus.cc:686] T 2a291bfa1f654e029f35a587d5b77131 P f4039ccfc39244e9821ed45c2ecdae76 [term 1 LEADER]: Becoming Leader. State: Replica: f4039ccfc39244e9821ed45c2ecdae76, State: Running, Role: LEADER
I20260812 02:41:15.933962 29629 consensus_queue.cc:237] T 2a291bfa1f654e029f35a587d5b77131 P f4039ccfc39244e9821ed45c2ecdae76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b12370667f144f5aa0495277e20e616" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 39705 } } peers { permanent_uuid: "3495979a70704c0c9cd6714d8238647a" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 42221 } } peers { permanent_uuid: "f4039ccfc39244e9821ed45c2ecdae76" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 33769 } }
I20260812 02:41:15.939177 29077 catalog_manager.cc:5495] T 2a291bfa1f654e029f35a587d5b77131 P f4039ccfc39244e9821ed45c2ecdae76 reported cstate change: term changed from 0 to 1, leader changed from <none> to f4039ccfc39244e9821ed45c2ecdae76 (127.7.132.130). New cstate: current_term: 1 leader_uuid: "f4039ccfc39244e9821ed45c2ecdae76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b12370667f144f5aa0495277e20e616" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 39705 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3495979a70704c0c9cd6714d8238647a" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 42221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f4039ccfc39244e9821ed45c2ecdae76" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 33769 } health_report { overall_health: HEALTHY } } }
W20260812 02:41:15.947047 29603 tablet.cc:2369] T 2a291bfa1f654e029f35a587d5b77131 P 7b12370667f144f5aa0495277e20e616: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 02:41:15.958415 29454 tablet.cc:2369] T 2a291bfa1f654e029f35a587d5b77131 P f4039ccfc39244e9821ed45c2ecdae76: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 02:41:15.964565 29619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:15.959217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33652 (local address 127.7.132.130:33769)
0812 02:41:15.959458 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0812 02:41:15.959462 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 02:41:15.959480 (+ 18us) server_negotiation.cc:407] Connection header received
0812 02:41:15.959670 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:41:15.959674 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:41:15.959766 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:41:15.959890 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 02:41:15.960210 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:15.960883 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:15.962405 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:15.962808 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:15.962901 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:15.963440 (+ 539us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 02:41:15.963951 (+ 511us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 02:41:15.964071 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0812 02:41:15.964161 (+ 90us) server_negotiation.cc:299] Negotiation successful
0812 02:41:15.964225 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260812 02:41:15.973449 29245 raft_consensus.cc:1261] T 2a291bfa1f654e029f35a587d5b77131 P 3495979a70704c0c9cd6714d8238647a [term 1 FOLLOWER]: Refusing update from remote peer f4039ccfc39244e9821ed45c2ecdae76: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:41:15.973448 29549 raft_consensus.cc:1261] T 2a291bfa1f654e029f35a587d5b77131 P 7b12370667f144f5aa0495277e20e616 [term 1 FOLLOWER]: Refusing update from remote peer f4039ccfc39244e9821ed45c2ecdae76: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:41:15.974782 29635 consensus_queue.cc:1035] T 2a291bfa1f654e029f35a587d5b77131 P f4039ccfc39244e9821ed45c2ecdae76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7b12370667f144f5aa0495277e20e616" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 39705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 02:41:15.975207 29299 tablet.cc:2369] T 2a291bfa1f654e029f35a587d5b77131 P 3495979a70704c0c9cd6714d8238647a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 02:41:15.975272 29629 consensus_queue.cc:1035] T 2a291bfa1f654e029f35a587d5b77131 P f4039ccfc39244e9821ed45c2ecdae76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3495979a70704c0c9cd6714d8238647a" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 42221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:41:16.005268 29077 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51736:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 02:41:16.005589 29077 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51736:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 02:41:16.009527 29077 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 23dea4a07da149b5b72720ae55fe1981: Sending DeleteTablet for 3 replicas of tablet 2a291bfa1f654e029f35a587d5b77131
I20260812 02:41:16.010772 29525 tablet_service.cc:1514] Processing DeleteTablet for tablet 2a291bfa1f654e029f35a587d5b77131 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:41:16 UTC) from {username='slave'} at 127.0.0.1:50344
I20260812 02:41:16.010874 29377 tablet_service.cc:1514] Processing DeleteTablet for tablet 2a291bfa1f654e029f35a587d5b77131 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:41:16 UTC) from {username='slave'} at 127.0.0.1:33648
I20260812 02:41:16.010989 29220 tablet_service.cc:1514] Processing DeleteTablet for tablet 2a291bfa1f654e029f35a587d5b77131 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:41:16 UTC) from {username='slave'} at 127.0.0.1:56734
I20260812 02:41:16.017176 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 29148
I20260812 02:41:16.024312 29649 tablet_replica.cc:330] T 2a291bfa1f654e029f35a587d5b77131 P 7b12370667f144f5aa0495277e20e616: stopping tablet replica
I20260812 02:41:16.024926 29649 raft_consensus.cc:2229] T 2a291bfa1f654e029f35a587d5b77131 P 7b12370667f144f5aa0495277e20e616 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 02:41:16.025606 29649 raft_consensus.cc:2258] T 2a291bfa1f654e029f35a587d5b77131 P 7b12370667f144f5aa0495277e20e616 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:41:16.026387 29650 tablet_replica.cc:330] T 2a291bfa1f654e029f35a587d5b77131 P f4039ccfc39244e9821ed45c2ecdae76: stopping tablet replica
I20260812 02:41:16.026684 29651 tablet_replica.cc:330] T 2a291bfa1f654e029f35a587d5b77131 P 3495979a70704c0c9cd6714d8238647a: stopping tablet replica
I20260812 02:41:16.026950 29650 raft_consensus.cc:2229] T 2a291bfa1f654e029f35a587d5b77131 P f4039ccfc39244e9821ed45c2ecdae76 [term 1 LEADER]: Raft consensus shutting down.
I20260812 02:41:16.027135 29651 raft_consensus.cc:2229] T 2a291bfa1f654e029f35a587d5b77131 P 3495979a70704c0c9cd6714d8238647a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 02:41:16.027690 29651 raft_consensus.cc:2258] T 2a291bfa1f654e029f35a587d5b77131 P 3495979a70704c0c9cd6714d8238647a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:41:16.027727 29650 raft_consensus.cc:2258] T 2a291bfa1f654e029f35a587d5b77131 P f4039ccfc39244e9821ed45c2ecdae76 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:41:16.028137 29649 ts_tablet_manager.cc:1905] T 2a291bfa1f654e029f35a587d5b77131 P 7b12370667f144f5aa0495277e20e616: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:41:16.029943 29651 ts_tablet_manager.cc:1905] T 2a291bfa1f654e029f35a587d5b77131 P 3495979a70704c0c9cd6714d8238647a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:41:16.029965 29650 ts_tablet_manager.cc:1905] T 2a291bfa1f654e029f35a587d5b77131 P f4039ccfc39244e9821ed45c2ecdae76: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:41:16.035351 29649 ts_tablet_manager.cc:1918] T 2a291bfa1f654e029f35a587d5b77131 P 7b12370667f144f5aa0495277e20e616: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 02:41:16.035578 29649 log.cc:1196] T 2a291bfa1f654e029f35a587d5b77131 P 7b12370667f144f5aa0495277e20e616: Deleting WAL directory at /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502156971023-7698-0/minicluster-data/ts-2/wal/wals/2a291bfa1f654e029f35a587d5b77131
I20260812 02:41:16.036176 29649 ts_tablet_manager.cc:1939] T 2a291bfa1f654e029f35a587d5b77131 P 7b12370667f144f5aa0495277e20e616: Deleting consensus metadata
I20260812 02:41:16.037197 29650 ts_tablet_manager.cc:1918] T 2a291bfa1f654e029f35a587d5b77131 P f4039ccfc39244e9821ed45c2ecdae76: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 02:41:16.037393 29650 log.cc:1196] T 2a291bfa1f654e029f35a587d5b77131 P f4039ccfc39244e9821ed45c2ecdae76: Deleting WAL directory at /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502156971023-7698-0/minicluster-data/ts-1/wal/wals/2a291bfa1f654e029f35a587d5b77131
I20260812 02:41:16.037736 29651 ts_tablet_manager.cc:1918] T 2a291bfa1f654e029f35a587d5b77131 P 3495979a70704c0c9cd6714d8238647a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 02:41:16.037940 29650 ts_tablet_manager.cc:1939] T 2a291bfa1f654e029f35a587d5b77131 P f4039ccfc39244e9821ed45c2ecdae76: Deleting consensus metadata
I20260812 02:41:16.037958 29651 log.cc:1196] T 2a291bfa1f654e029f35a587d5b77131 P 3495979a70704c0c9cd6714d8238647a: Deleting WAL directory at /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786502156971023-7698-0/minicluster-data/ts-0/wal/wals/2a291bfa1f654e029f35a587d5b77131
I20260812 02:41:16.038172 29062 catalog_manager.cc:4841] TS 7b12370667f144f5aa0495277e20e616 (127.7.132.131:39705): tablet 2a291bfa1f654e029f35a587d5b77131 (table test-table [id=3a21022f6d0f4aeb9fc2a5b02df23ebe]) successfully deleted
I20260812 02:41:16.038754 29651 ts_tablet_manager.cc:1939] T 2a291bfa1f654e029f35a587d5b77131 P 3495979a70704c0c9cd6714d8238647a: Deleting consensus metadata
I20260812 02:41:16.040335 29059 catalog_manager.cc:4841] TS f4039ccfc39244e9821ed45c2ecdae76 (127.7.132.130:33769): tablet 2a291bfa1f654e029f35a587d5b77131 (table test-table [id=3a21022f6d0f4aeb9fc2a5b02df23ebe]) successfully deleted
I20260812 02:41:16.040436 29061 catalog_manager.cc:4841] TS 3495979a70704c0c9cd6714d8238647a (127.7.132.129:42221): tablet 2a291bfa1f654e029f35a587d5b77131 (table test-table [id=3a21022f6d0f4aeb9fc2a5b02df23ebe]) successfully deleted
I20260812 02:41:16.044404 29617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:16.018711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56748 (local address 127.7.132.129:42221)
0812 02:41:16.019594 (+ 883us) server_negotiation.cc:206] Beginning negotiation
0812 02:41:16.019604 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 02:41:16.034163 (+ 14559us) server_negotiation.cc:407] Connection header received
0812 02:41:16.034495 (+ 332us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:41:16.034502 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:41:16.034627 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:41:16.034704 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:41:16.035399 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:16.036598 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:16.040872 (+ 4274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:16.041501 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:16.041634 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:16.042788 (+ 1154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:41:16.042799 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:41:16.042817 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:41:16.043592 (+ 775us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:41:16.043653 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:41:16.043678 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:41:16.043808 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0812 02:41:16.043925 (+ 117us) server_negotiation.cc:299] Negotiation successful
0812 02:41:16.044052 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":697}
I20260812 02:41:16.045167 29292 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:41:16.134361 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 29148
I20260812 02:41:16.157238 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 29303
I20260812 02:41:16.166239 29619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:16.158228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33654 (local address 127.7.132.130:33769)
0812 02:41:16.158425 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0812 02:41:16.158431 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:41:16.158453 (+ 22us) server_negotiation.cc:407] Connection header received
0812 02:41:16.158571 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:41:16.158577 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:41:16.158684 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:41:16.158797 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:41:16.159232 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:16.160450 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:16.162408 (+ 1958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:16.163081 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:16.163218 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:16.164450 (+ 1232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:41:16.164462 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:41:16.164482 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:41:16.165338 (+ 856us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:41:16.165402 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:41:16.165427 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:41:16.165539 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0812 02:41:16.165698 (+ 159us) server_negotiation.cc:299] Negotiation successful
0812 02:41:16.165812 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260812 02:41:16.167058 29447 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:41:16.279096 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 29303
I20260812 02:41:16.302881 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 29456
I20260812 02:41:16.311100 29616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:16.303795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49362 (local address 127.7.132.131:39705)
0812 02:41:16.304037 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0812 02:41:16.304046 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 02:41:16.304212 (+ 166us) server_negotiation.cc:407] Connection header received
0812 02:41:16.304454 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:41:16.304470 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:41:16.304579 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:41:16.304698 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:41:16.305168 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:16.306022 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:16.307721 (+ 1699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:16.308132 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:16.308274 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:16.309466 (+ 1192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:41:16.309475 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:41:16.309490 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:41:16.310290 (+ 800us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:41:16.310352 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:41:16.310374 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:41:16.310501 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0812 02:41:16.310655 (+ 154us) server_negotiation.cc:299] Negotiation successful
0812 02:41:16.310768 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 02:41:16.311753 29597 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:41:16.402472 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 29456
I20260812 02:41:16.424567 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 29041
I20260812 02:41:16.425915 29116 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:41:16.540298 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 29041
2026-08-12T02:41:16Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (2947 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502156971023-7698-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 02:41:16 dist-test-slave-mn4r krb5kdc[29661](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 02:41:16 dist-test-slave-mn4r krb5kdc[29661](info): set up 2 sockets
Aug 12 02:41:16 dist-test-slave-mn4r krb5kdc[29661](info): commencing operation
krb5kdc: starting...
W20260812 02:41:17.847710 7698 mini_kdc.cc:121] Time spent starting KDC: real 1.259s user 0.000s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502156971023-7698-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502156971023-7698-0/krb5kdc/test-user.keytab.
Aug 12 02:41:18 dist-test-slave-mn4r krb5kdc[29661](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502478, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T02:41:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T02:41:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.132.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
I20260812 02:41:18.163760 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502156971023-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502156971023-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502156971023-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:38321
--webserver_interface=127.7.132.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:46573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.132.190:38321
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:41:18.508881 29676 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:41:18.509310 29676 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:41:18.509490 29676 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:41:18.516988 29676 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:41:18.517153 29676 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:41:18.517277 29676 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:41:18.517393 29676 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:41:18.517474 29676 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:41:18.527557 29676 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:46573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502156971023-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502156971023-7698-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.132.190:38321
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:38321
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.132.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502156971023-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:41:18.529345 29676 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:41:18.531165 29676 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:41:18.543434 29684 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:41:18.543473 29681 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:41:18.544157 29682 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:41:18.544251 29676 server_base.cc:1034] running on GCE node
I20260812 02:41:18.545173 29676 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:41:18.546751 29676 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:41:18.547991 29676 hybrid_clock.cc:648] HybridClock initialized: now 1786502478547928 us; error 46 us; skew 500 ppm
I20260812 02:41:18.548360 29676 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:41:18 dist-test-slave-mn4r krb5kdc[29661](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502478, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:41:18.551719 29676 init.cc:377] Logged in from keytab as kudu/127.7.132.190@KRBTEST.COM (short username kudu)
I20260812 02:41:18.553229 29676 webserver.cc:460] Webserver started at http://127.7.132.190:37147/ using document root <none> and password file <none>
I20260812 02:41:18.553865 29676 fs_manager.cc:359] Metadata directory not provided
I20260812 02:41:18.553984 29676 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:41:18.554334 29676 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:41:18.557281 29676 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502156971023-7698-0/minicluster-data/master-0/data/instance:
uuid: "f9442c6a8c994b7dba3179b01e1dfcd0"
format_stamp: "Formatted at 2026-08-12 02:41:18 on dist-test-slave-mn4r"
server_key: "54210c318614658afb870e73e7eeb799"
server_key_iv: "af5d1f39b4f680e689d3f045a2a35e9a"
server_key_version: "encryptionkey@0"
I20260812 02:41:18.557991 29676 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502156971023-7698-0/minicluster-data/master-0/wal/instance:
uuid: "f9442c6a8c994b7dba3179b01e1dfcd0"
format_stamp: "Formatted at 2026-08-12 02:41:18 on dist-test-slave-mn4r"
server_key: "54210c318614658afb870e73e7eeb799"
server_key_iv: "af5d1f39b4f680e689d3f045a2a35e9a"
server_key_version: "encryptionkey@0"
I20260812 02:41:18.563318 29676 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260812 02:41:18.566787 29691 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:41:18.568150 29676 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 02:41:18.568342 29676 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502156971023-7698-0/minicluster-data/master-0/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502156971023-7698-0/minicluster-data/master-0/wal
uuid: "f9442c6a8c994b7dba3179b01e1dfcd0"
format_stamp: "Formatted at 2026-08-12 02:41:18 on dist-test-slave-mn4r"
server_key: "54210c318614658afb870e73e7eeb799"
server_key_iv: "af5d1f39b4f680e689d3f045a2a35e9a"
server_key_version: "encryptionkey@0"
I20260812 02:41:18.568553 29676 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502156971023-7698-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502156971023-7698-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502156971023-7698-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:41:18.613692 29676 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:41:18.618579 29676 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:41:18.618932 29676 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:41:18.638617 29676 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.190:38321
I20260812 02:41:18.638665 29749 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.190:38321 every 8 connection(s)
I20260812 02:41:18.640173 29676 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
I20260812 02:41:18.644227 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 29676
I20260812 02:41:18.644466 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502156971023-7698-0/minicluster-data/master-0/wal/instance
I20260812 02:41:18.644804 29750 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:41:18.645089 7698 external_mini_cluster.cc:1428] Setting key 7e0b261bac3e4fa0d1ad2459cdc49db3
I20260812 02:41:18.658507 29750 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9442c6a8c994b7dba3179b01e1dfcd0: Bootstrap starting.
Aug 12 02:41:18 dist-test-slave-mn4r krb5kdc[29661](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502478, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 02:41:18.661849 29750 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f9442c6a8c994b7dba3179b01e1dfcd0: Neither blocks nor log segments found. Creating new log.
I20260812 02:41:18.663040 29750 log.cc:824] T 00000000000000000000000000000000 P f9442c6a8c994b7dba3179b01e1dfcd0: Log is configured to *not* fsync() on all Append() calls
I20260812 02:41:18.666355 29750 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9442c6a8c994b7dba3179b01e1dfcd0: No bootstrap required, opened a new log
I20260812 02:41:18.667846 29753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:18.647051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54290 (local address 127.7.132.190:38321)
0812 02:41:18.647867 (+ 816us) server_negotiation.cc:206] Beginning negotiation
0812 02:41:18.647880 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 02:41:18.647921 (+ 41us) server_negotiation.cc:407] Connection header received
0812 02:41:18.649541 (+ 1620us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:41:18.649564 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:41:18.649957 (+ 393us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:41:18.650463 (+ 506us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:41:18.652000 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:18.653344 (+ 1344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:18.654750 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:18.655470 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:18.655687 (+ 217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:18.660310 (+ 4623us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:41:18.660319 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:41:18.660333 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:41:18.660373 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:41:18.663703 (+ 3330us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:41:18.664468 (+ 765us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:41:18.664474 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:41:18.664480 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:41:18.664651 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:41:18.665075 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:41:18.665078 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:41:18.665080 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:41:18.665580 (+ 500us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:41:18.665797 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0812 02:41:18.666358 (+ 561us) server_negotiation.cc:299] Negotiation successful
0812 02:41:18.666811 (+ 453us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":602,"thread_start_us":378,"threads_started":1}
I20260812 02:41:18.674355 29750 raft_consensus.cc:348] T 00000000000000000000000000000000 P f9442c6a8c994b7dba3179b01e1dfcd0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9442c6a8c994b7dba3179b01e1dfcd0" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 38321 } }
I20260812 02:41:18.674752 29750 raft_consensus.cc:374] T 00000000000000000000000000000000 P f9442c6a8c994b7dba3179b01e1dfcd0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:41:18.674875 29750 raft_consensus.cc:729] T 00000000000000000000000000000000 P f9442c6a8c994b7dba3179b01e1dfcd0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9442c6a8c994b7dba3179b01e1dfcd0, State: Initialized, Role: FOLLOWER
I20260812 02:41:18.675467 29750 consensus_queue.cc:260] T 00000000000000000000000000000000 P f9442c6a8c994b7dba3179b01e1dfcd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9442c6a8c994b7dba3179b01e1dfcd0" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 38321 } }
I20260812 02:41:18.675709 29750 raft_consensus.cc:388] T 00000000000000000000000000000000 P f9442c6a8c994b7dba3179b01e1dfcd0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 02:41:18.675854 29750 raft_consensus.cc:482] T 00000000000000000000000000000000 P f9442c6a8c994b7dba3179b01e1dfcd0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 02:41:18.676064 29750 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f9442c6a8c994b7dba3179b01e1dfcd0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:41:18.678334 29750 raft_consensus.cc:504] T 00000000000000000000000000000000 P f9442c6a8c994b7dba3179b01e1dfcd0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9442c6a8c994b7dba3179b01e1dfcd0" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 38321 } }
I20260812 02:41:18.678866 29750 leader_election.cc:302] T 00000000000000000000000000000000 P f9442c6a8c994b7dba3179b01e1dfcd0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f9442c6a8c994b7dba3179b01e1dfcd0; no voters:
I20260812 02:41:18.679492 29750 leader_election.cc:288] T 00000000000000000000000000000000 P f9442c6a8c994b7dba3179b01e1dfcd0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 02:41:18.679664 29755 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f9442c6a8c994b7dba3179b01e1dfcd0 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:41:18.680253 29755 raft_consensus.cc:686] T 00000000000000000000000000000000 P f9442c6a8c994b7dba3179b01e1dfcd0 [term 1 LEADER]: Becoming Leader. State: Replica: f9442c6a8c994b7dba3179b01e1dfcd0, State: Running, Role: LEADER
I20260812 02:41:18.680894 29755 consensus_queue.cc:237] T 00000000000000000000000000000000 P f9442c6a8c994b7dba3179b01e1dfcd0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9442c6a8c994b7dba3179b01e1dfcd0" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 38321 } }
I20260812 02:41:18.682546 29750 sys_catalog.cc:564] T 00000000000000000000000000000000 P f9442c6a8c994b7dba3179b01e1dfcd0 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:41:18.683892 29756 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9442c6a8c994b7dba3179b01e1dfcd0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f9442c6a8c994b7dba3179b01e1dfcd0. Latest consensus state: current_term: 1 leader_uuid: "f9442c6a8c994b7dba3179b01e1dfcd0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9442c6a8c994b7dba3179b01e1dfcd0" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 38321 } } }
I20260812 02:41:18.683949 29757 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9442c6a8c994b7dba3179b01e1dfcd0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f9442c6a8c994b7dba3179b01e1dfcd0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9442c6a8c994b7dba3179b01e1dfcd0" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 38321 } } }
I20260812 02:41:18.684419 29757 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9442c6a8c994b7dba3179b01e1dfcd0 [sys.catalog]: This master's current role is: LEADER
I20260812 02:41:18.684397 29756 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9442c6a8c994b7dba3179b01e1dfcd0 [sys.catalog]: This master's current role is: LEADER
I20260812 02:41:18.686239 29760 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:41:18.691555 29760 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 02:41:18.701767 29760 catalog_manager.cc:1347] Generated new cluster ID: f7ca3357dd694b1bbc09c41bf379bdd4
I20260812 02:41:18.701906 29760 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 02:41:18.712426 29760 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 02:41:18.714246 29760 catalog_manager.cc:1504] Loading token signing keys...
I20260812 02:41:18.722635 29760 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f9442c6a8c994b7dba3179b01e1dfcd0: Generated new TSK 0
I20260812 02:41:18.723423 29760 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.132.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.129.keytab.
Entry for principal kudu/127.7.132.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.129.keytab.
I20260812 02:41:18.793121 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502156971023-7698-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502156971023-7698-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502156971023-7698-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502156971023-7698-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.129:0
--local_ip_for_outbound_sockets=127.7.132.129
--webserver_interface=127.7.132.129
--webserver_port=0
--tserver_master_addrs=127.7.132.190:38321
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.129.keytab
--principal=kudu/127.7.132.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:46573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:41:19.143906 29778 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:41:19.144178 29778 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:41:19.144381 29778 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:41:19.151460 29778 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:41:19.151592 29778 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:41:19.151790 29778 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.129
I20260812 02:41:19.161387 29778 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:46573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502156971023-7698-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502156971023-7698-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.129.keytab
--principal=kudu/127.7.132.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502156971023-7698-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.132.129
--webserver_port=0
--tserver_master_addrs=127.7.132.190:38321
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.129
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502156971023-7698-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:41:19.162818 29778 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:41:19.164156 29778 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:41:19.178789 29784 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:41:19.178702 29787 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:41:19.178733 29785 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:41:19.179526 29778 server_base.cc:1034] running on GCE node
I20260812 02:41:19.180160 29778 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:41:19.181141 29778 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:41:19.182333 29778 hybrid_clock.cc:648] HybridClock initialized: now 1786502479182278 us; error 54 us; skew 500 ppm
I20260812 02:41:19.182979 29778 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:41:19 dist-test-slave-mn4r krb5kdc[29661](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502479, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:41:19.192299 29778 init.cc:377] Logged in from keytab as kudu/127.7.132.129@KRBTEST.COM (short username kudu)
I20260812 02:41:19.193857 29778 webserver.cc:460] Webserver started at http://127.7.132.129:44037/ using document root <none> and password file <none>
I20260812 02:41:19.194628 29778 fs_manager.cc:359] Metadata directory not provided
I20260812 02:41:19.194761 29778 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:41:19.195096 29778 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:41:19.198166 29778 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502156971023-7698-0/minicluster-data/ts-0/data/instance:
uuid: "edf58a468a2843b393738669a59ae39f"
format_stamp: "Formatted at 2026-08-12 02:41:19 on dist-test-slave-mn4r"
server_key: "9f5e5904dc0f8a2aa216f32ec03707e3"
server_key_iv: "416d78d205b72986fe35deecb4d7d5c7"
server_key_version: "encryptionkey@0"
I20260812 02:41:19.198977 29778 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502156971023-7698-0/minicluster-data/ts-0/wal/instance:
uuid: "edf58a468a2843b393738669a59ae39f"
format_stamp: "Formatted at 2026-08-12 02:41:19 on dist-test-slave-mn4r"
server_key: "9f5e5904dc0f8a2aa216f32ec03707e3"
server_key_iv: "416d78d205b72986fe35deecb4d7d5c7"
server_key_version: "encryptionkey@0"
I20260812 02:41:19.204341 29778 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260812 02:41:19.207774 29797 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:41:19.209040 29778 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 02:41:19.209216 29778 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502156971023-7698-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502156971023-7698-0/minicluster-data/ts-0/wal
uuid: "edf58a468a2843b393738669a59ae39f"
format_stamp: "Formatted at 2026-08-12 02:41:19 on dist-test-slave-mn4r"
server_key: "9f5e5904dc0f8a2aa216f32ec03707e3"
server_key_iv: "416d78d205b72986fe35deecb4d7d5c7"
server_key_version: "encryptionkey@0"
I20260812 02:41:19.209419 29778 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502156971023-7698-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502156971023-7698-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502156971023-7698-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:41:19.227002 29778 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:41:19.231536 29778 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:41:19.231844 29778 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:41:19.232863 29778 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:41:19.234338 29778 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:41:19.234413 29778 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:41:19.234589 29778 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:41:19.234644 29778 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:41:19.268702 29778 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.129:38225
I20260812 02:41:19.268756 29921 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.129:38225 every 8 connection(s)
I20260812 02:41:19.270121 29778 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502156971023-7698-0/minicluster-data/ts-0/data/info.pb
I20260812 02:41:19.276932 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 29778
I20260812 02:41:19.277148 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502156971023-7698-0/minicluster-data/ts-0/wal/instance
I20260812 02:41:19.277760 7698 external_mini_cluster.cc:1428] Setting key b574732ef625a000883cd904ea1d2dc9
Aug 12 02:41:19 dist-test-slave-mn4r krb5kdc[29661](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502479, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.129@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 02:41:19.291975 29924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:19.273093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:49261 (local address 127.7.132.190:38321)
0812 02:41:19.273646 (+ 553us) server_negotiation.cc:206] Beginning negotiation
0812 02:41:19.273653 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:41:19.274727 (+ 1074us) server_negotiation.cc:407] Connection header received
0812 02:41:19.276680 (+ 1953us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:41:19.276685 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:41:19.276757 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:41:19.276856 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:41:19.278969 (+ 2113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:19.279878 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:19.281485 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:19.281898 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:19.282045 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:19.285817 (+ 3772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:41:19.285822 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:41:19.285826 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:41:19.285859 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:41:19.288787 (+ 2928us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:41:19.289535 (+ 748us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:41:19.289548 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:41:19.289550 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:41:19.289673 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:41:19.290146 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:41:19.290158 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:41:19.290160 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:41:19.290427 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:41:19.290588 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0812 02:41:19.291347 (+ 759us) server_negotiation.cc:299] Negotiation successful
0812 02:41:19.291580 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":417,"thread_start_us":291,"threads_started":1}
I20260812 02:41:19.293097 29925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:19.273761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:38321 (local address 127.7.132.129:49261)
0812 02:41:19.274542 (+ 781us) negotiation.cc:107] Waiting for socket to connect
0812 02:41:19.274589 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 02:41:19.276335 (+ 1746us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:41:19.277173 (+ 838us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:41:19.277186 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:41:19.277999 (+ 813us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:41:19.278778 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:41:19.278798 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:41:19.280063 (+ 1265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:41:19.280067 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:41:19.281325 (+ 1258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:41:19.281338 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:41:19.282186 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:41:19.282194 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:41:19.282437 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:19.283142 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:41:19.283165 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:41:19.285592 (+ 2427us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:41:19.288940 (+ 3348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:41:19.288949 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:41:19.288966 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:41:19.289398 (+ 432us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:41:19.289789 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:41:19.289794 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:41:19.289798 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:41:19.289997 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:41:19.290571 (+ 574us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:41:19.290578 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:41:19.290939 (+ 361us) client_negotiation.cc:769] Sending connection context
0812 02:41:19.291224 (+ 285us) client_negotiation.cc:240] Negotiation successful
0812 02:41:19.291792 (+ 568us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":547,"thread_start_us":349,"threads_started":1}
I20260812 02:41:19.294826 29922 heartbeater.cc:347] Connected to a master server at 127.7.132.190:38321
I20260812 02:41:19.295178 29922 heartbeater.cc:464] Registering TS with master...
I20260812 02:41:19.295991 29922 heartbeater.cc:511] Master 127.7.132.190:38321 requested a full tablet report, sending...
I20260812 02:41:19.298509 29709 ts_manager.cc:194] Registered new tserver with Master: edf58a468a2843b393738669a59ae39f (127.7.132.129:38225)
I20260812 02:41:19.300740 29709 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.129@KRBTEST.COM'} at 127.7.132.129:49261
WARNING: no policy specified for kudu/127.7.132.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.130.keytab.
Entry for principal kudu/127.7.132.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.130.keytab.
I20260812 02:41:19.355314 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502156971023-7698-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502156971023-7698-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502156971023-7698-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502156971023-7698-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.130:0
--local_ip_for_outbound_sockets=127.7.132.130
--webserver_interface=127.7.132.130
--webserver_port=0
--tserver_master_addrs=127.7.132.190:38321
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.130.keytab
--principal=kudu/127.7.132.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:46573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:41:19.713272 29928 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:41:19.713549 29928 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:41:19.713747 29928 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:41:19.721673 29928 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:41:19.721800 29928 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:41:19.722003 29928 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.130
I20260812 02:41:19.731895 29928 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:46573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502156971023-7698-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502156971023-7698-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.130.keytab
--principal=kudu/127.7.132.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502156971023-7698-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.132.130
--webserver_port=0
--tserver_master_addrs=127.7.132.190:38321
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.130
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502156971023-7698-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:41:19.733237 29928 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:41:19.734690 29928 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:41:19.747949 29936 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:41:19.747924 29935 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:41:19.748143 29938 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:41:19.748659 29928 server_base.cc:1034] running on GCE node
I20260812 02:41:19.749372 29928 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:41:19.750361 29928 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:41:19.751592 29928 hybrid_clock.cc:648] HybridClock initialized: now 1786502479751556 us; error 48 us; skew 500 ppm
I20260812 02:41:19.751909 29928 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:41:19 dist-test-slave-mn4r krb5kdc[29661](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502479, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:41:19.755050 29928 init.cc:377] Logged in from keytab as kudu/127.7.132.130@KRBTEST.COM (short username kudu)
I20260812 02:41:19.756448 29928 webserver.cc:460] Webserver started at http://127.7.132.130:34467/ using document root <none> and password file <none>
I20260812 02:41:19.757135 29928 fs_manager.cc:359] Metadata directory not provided
I20260812 02:41:19.757280 29928 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:41:19.757606 29928 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:41:19.760555 29928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502156971023-7698-0/minicluster-data/ts-1/data/instance:
uuid: "7d88fea233a24b47870378b942d168ea"
format_stamp: "Formatted at 2026-08-12 02:41:19 on dist-test-slave-mn4r"
server_key: "2484df7372e2e1daddfa573a37cc88a2"
server_key_iv: "170549e76a4d009904c2c8fa620537d6"
server_key_version: "encryptionkey@0"
I20260812 02:41:19.761329 29928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502156971023-7698-0/minicluster-data/ts-1/wal/instance:
uuid: "7d88fea233a24b47870378b942d168ea"
format_stamp: "Formatted at 2026-08-12 02:41:19 on dist-test-slave-mn4r"
server_key: "2484df7372e2e1daddfa573a37cc88a2"
server_key_iv: "170549e76a4d009904c2c8fa620537d6"
server_key_version: "encryptionkey@0"
I20260812 02:41:19.766539 29928 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260812 02:41:19.770372 29947 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:41:19.771785 29928 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 02:41:19.771982 29928 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502156971023-7698-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502156971023-7698-0/minicluster-data/ts-1/wal
uuid: "7d88fea233a24b47870378b942d168ea"
format_stamp: "Formatted at 2026-08-12 02:41:19 on dist-test-slave-mn4r"
server_key: "2484df7372e2e1daddfa573a37cc88a2"
server_key_iv: "170549e76a4d009904c2c8fa620537d6"
server_key_version: "encryptionkey@0"
I20260812 02:41:19.772241 29928 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502156971023-7698-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502156971023-7698-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502156971023-7698-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:41:19.793393 29928 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:41:19.797406 29928 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:41:19.797721 29928 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:41:19.798790 29928 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:41:19.800173 29928 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:41:19.800288 29928 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:41:19.800436 29928 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:41:19.800508 29928 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:41:19.834036 29928 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.130:36693
I20260812 02:41:19.834128 30081 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.130:36693 every 8 connection(s)
I20260812 02:41:19.835425 29928 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502156971023-7698-0/minicluster-data/ts-1/data/info.pb
I20260812 02:41:19.837646 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 29928
I20260812 02:41:19.837821 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502156971023-7698-0/minicluster-data/ts-1/wal/instance
I20260812 02:41:19.838433 7698 external_mini_cluster.cc:1428] Setting key 0eaef55958c8cbf0f7d07d101de6a288
Aug 12 02:41:19 dist-test-slave-mn4r krb5kdc[29661](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502479, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.130@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 02:41:19.860527 30084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:19.838249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:51327 (local address 127.7.132.190:38321)
0812 02:41:19.838782 (+ 533us) server_negotiation.cc:206] Beginning negotiation
0812 02:41:19.838789 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:41:19.839956 (+ 1167us) server_negotiation.cc:407] Connection header received
0812 02:41:19.842630 (+ 2674us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:41:19.842636 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:41:19.842747 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:41:19.842869 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:41:19.845046 (+ 2177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:19.845746 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:19.848303 (+ 2557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:19.848749 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:19.848859 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:19.853452 (+ 4593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:41:19.853457 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:41:19.853460 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:41:19.853492 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:41:19.857069 (+ 3577us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:41:19.857787 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:41:19.857796 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:41:19.857802 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:41:19.857987 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:41:19.858465 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:41:19.858473 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:41:19.858479 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:41:19.858923 (+ 444us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:41:19.859086 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0812 02:41:19.859749 (+ 663us) server_negotiation.cc:299] Negotiation successful
0812 02:41:19.860031 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":402,"thread_start_us":288,"threads_started":1}
I20260812 02:41:19.861121 30085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:19.838895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:38321 (local address 127.7.132.130:51327)
0812 02:41:19.839808 (+ 913us) negotiation.cc:107] Waiting for socket to connect
0812 02:41:19.839847 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 02:41:19.842297 (+ 2450us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:41:19.843161 (+ 864us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:41:19.843176 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:41:19.844008 (+ 832us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:41:19.844855 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:41:19.844885 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:41:19.845898 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:41:19.845906 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:41:19.848059 (+ 2153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:41:19.848077 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:41:19.849004 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:41:19.849013 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:41:19.849347 (+ 334us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:19.850161 (+ 814us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:41:19.850200 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:41:19.853209 (+ 3009us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:41:19.857209 (+ 4000us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:41:19.857215 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:41:19.857228 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:41:19.857650 (+ 422us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:41:19.858147 (+ 497us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:41:19.858152 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:41:19.858154 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:41:19.858337 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:41:19.859048 (+ 711us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:41:19.859053 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:41:19.859363 (+ 310us) client_negotiation.cc:769] Sending connection context
0812 02:41:19.859583 (+ 220us) client_negotiation.cc:240] Negotiation successful
0812 02:41:19.859971 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":641,"thread_start_us":400,"threads_started":1}
I20260812 02:41:19.862844 30082 heartbeater.cc:347] Connected to a master server at 127.7.132.190:38321
I20260812 02:41:19.863173 30082 heartbeater.cc:464] Registering TS with master...
I20260812 02:41:19.863935 30082 heartbeater.cc:511] Master 127.7.132.190:38321 requested a full tablet report, sending...
I20260812 02:41:19.865495 29709 ts_manager.cc:194] Registered new tserver with Master: 7d88fea233a24b47870378b942d168ea (127.7.132.130:36693)
I20260812 02:41:19.866408 29709 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.130@KRBTEST.COM'} at 127.7.132.130:51327
WARNING: no policy specified for kudu/127.7.132.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.131.keytab.
Entry for principal kudu/127.7.132.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.131.keytab.
I20260812 02:41:19.911058 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502156971023-7698-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502156971023-7698-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502156971023-7698-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502156971023-7698-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.131:0
--local_ip_for_outbound_sockets=127.7.132.131
--webserver_interface=127.7.132.131
--webserver_port=0
--tserver_master_addrs=127.7.132.190:38321
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.131.keytab
--principal=kudu/127.7.132.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:46573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:41:20.258818 30090 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:41:20.259100 30090 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:41:20.259196 30090 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:41:20.266608 30090 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:41:20.266721 30090 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:41:20.266832 30090 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.131
I20260812 02:41:20.276964 30090 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:46573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502156971023-7698-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502156971023-7698-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.131.keytab
--principal=kudu/127.7.132.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502156971023-7698-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.132.131
--webserver_port=0
--tserver_master_addrs=127.7.132.190:38321
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.131
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502156971023-7698-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:41:20.278163 30090 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:41:20.279474 30090 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:41:20.293191 30097 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:41:20.293165 30099 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:41:20.293244 30096 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:41:20.293947 30090 server_base.cc:1034] running on GCE node
I20260812 02:41:20.294703 30090 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:41:20.295619 30090 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:41:20.296855 30090 hybrid_clock.cc:648] HybridClock initialized: now 1786502480296812 us; error 27 us; skew 500 ppm
I20260812 02:41:20.297242 30090 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:41:20 dist-test-slave-mn4r krb5kdc[29661](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502480, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:41:20.300465 30090 init.cc:377] Logged in from keytab as kudu/127.7.132.131@KRBTEST.COM (short username kudu)
I20260812 02:41:20.301870 30090 webserver.cc:460] Webserver started at http://127.7.132.131:42713/ using document root <none> and password file <none>
I20260812 02:41:20.302614 30090 fs_manager.cc:359] Metadata directory not provided
I20260812 02:41:20.302717 30090 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:41:20.302980 30090 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:41:20.303854 29922 heartbeater.cc:503] Master 127.7.132.190:38321 was elected leader, sending a full tablet report...
I20260812 02:41:20.306545 30090 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502156971023-7698-0/minicluster-data/ts-2/data/instance:
uuid: "081da6508f944603b63ee1f8472e7f73"
format_stamp: "Formatted at 2026-08-12 02:41:20 on dist-test-slave-mn4r"
server_key: "58ed5fad247f0f9981f5701cee9ddd26"
server_key_iv: "d4aaab7bb3cc31ba17d84b3ce74c7248"
server_key_version: "encryptionkey@0"
I20260812 02:41:20.307557 30090 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502156971023-7698-0/minicluster-data/ts-2/wal/instance:
uuid: "081da6508f944603b63ee1f8472e7f73"
format_stamp: "Formatted at 2026-08-12 02:41:20 on dist-test-slave-mn4r"
server_key: "58ed5fad247f0f9981f5701cee9ddd26"
server_key_iv: "d4aaab7bb3cc31ba17d84b3ce74c7248"
server_key_version: "encryptionkey@0"
I20260812 02:41:20.312947 30090 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260812 02:41:20.316509 30107 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:41:20.317828 30090 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260812 02:41:20.318002 30090 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502156971023-7698-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502156971023-7698-0/minicluster-data/ts-2/wal
uuid: "081da6508f944603b63ee1f8472e7f73"
format_stamp: "Formatted at 2026-08-12 02:41:20 on dist-test-slave-mn4r"
server_key: "58ed5fad247f0f9981f5701cee9ddd26"
server_key_iv: "d4aaab7bb3cc31ba17d84b3ce74c7248"
server_key_version: "encryptionkey@0"
I20260812 02:41:20.318289 30090 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502156971023-7698-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502156971023-7698-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502156971023-7698-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:41:20.336854 30090 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:41:20.340793 30090 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:41:20.341102 30090 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:41:20.342121 30090 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:41:20.343600 30090 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:41:20.343701 30090 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:41:20.343834 30090 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:41:20.343918 30090 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:41:20.377873 30090 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.131:36949
I20260812 02:41:20.377923 30238 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.131:36949 every 8 connection(s)
I20260812 02:41:20.379230 30090 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502156971023-7698-0/minicluster-data/ts-2/data/info.pb
I20260812 02:41:20.382032 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 30090
I20260812 02:41:20.382251 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502156971023-7698-0/minicluster-data/ts-2/wal/instance
I20260812 02:41:20.382887 7698 external_mini_cluster.cc:1428] Setting key 72c775870e5525b3abdf5a36c4b7f70c
Aug 12 02:41:20 dist-test-slave-mn4r krb5kdc[29661](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502480, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.131@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 02:41:20.400484 30242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:20.381769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:58883 (local address 127.7.132.190:38321)
0812 02:41:20.382317 (+ 548us) server_negotiation.cc:206] Beginning negotiation
0812 02:41:20.382322 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:41:20.383603 (+ 1281us) server_negotiation.cc:407] Connection header received
0812 02:41:20.385744 (+ 2141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:41:20.385748 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:41:20.385811 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:41:20.385940 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:41:20.387540 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:20.388450 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:20.390020 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:20.390463 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:20.390557 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:20.393777 (+ 3220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:41:20.393782 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:41:20.393786 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:41:20.393817 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:41:20.396530 (+ 2713us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:41:20.397309 (+ 779us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:41:20.397315 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:41:20.397318 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:41:20.397479 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:41:20.397948 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:41:20.397960 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:41:20.397963 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:41:20.398756 (+ 793us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:41:20.398982 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0812 02:41:20.399845 (+ 863us) server_negotiation.cc:299] Negotiation successful
0812 02:41:20.400077 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":446,"thread_start_us":321,"threads_started":1}
I20260812 02:41:20.401553 30243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:20.382576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:38321 (local address 127.7.132.131:58883)
0812 02:41:20.383386 (+ 810us) negotiation.cc:107] Waiting for socket to connect
0812 02:41:20.383449 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0812 02:41:20.385441 (+ 1992us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:41:20.386207 (+ 766us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:41:20.386215 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:41:20.386766 (+ 551us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:41:20.387395 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:41:20.387414 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:41:20.388603 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:41:20.388606 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:41:20.389897 (+ 1291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:41:20.389909 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:41:20.390659 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:41:20.390664 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:41:20.390851 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:20.391455 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:41:20.391482 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:41:20.393582 (+ 2100us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:41:20.396686 (+ 3104us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:41:20.396693 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:41:20.396704 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:41:20.397195 (+ 491us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:41:20.397602 (+ 407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:41:20.397610 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:41:20.397616 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:41:20.397820 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:41:20.398917 (+ 1097us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:41:20.398927 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:41:20.399430 (+ 503us) client_negotiation.cc:769] Sending connection context
0812 02:41:20.399742 (+ 312us) client_negotiation.cc:240] Negotiation successful
0812 02:41:20.400277 (+ 535us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":528,"thread_start_us":313,"threads_started":1}
I20260812 02:41:20.402961 30240 heartbeater.cc:347] Connected to a master server at 127.7.132.190:38321
I20260812 02:41:20.403249 30240 heartbeater.cc:464] Registering TS with master...
I20260812 02:41:20.403921 30240 heartbeater.cc:511] Master 127.7.132.190:38321 requested a full tablet report, sending...
I20260812 02:41:20.405299 29709 ts_manager.cc:194] Registered new tserver with Master: 081da6508f944603b63ee1f8472e7f73 (127.7.132.131:36949)
I20260812 02:41:20.406049 29709 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.131@KRBTEST.COM'} at 127.7.132.131:58883
I20260812 02:41:20.410449 7698 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 02:41:20.427829 30242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:20.414671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54292 (local address 127.7.132.190:38321)
0812 02:41:20.414832 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0812 02:41:20.414836 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 02:41:20.415307 (+ 471us) server_negotiation.cc:407] Connection header received
0812 02:41:20.415440 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:41:20.415444 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:41:20.415511 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:41:20.415598 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:41:20.416975 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:20.417807 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:20.419169 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:20.419609 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:20.419714 (+ 105us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:20.422643 (+ 2929us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:41:20.422648 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:41:20.422652 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:41:20.422683 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:41:20.425122 (+ 2439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:41:20.425906 (+ 784us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:41:20.425909 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:41:20.425912 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:41:20.426040 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:41:20.426493 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:41:20.426496 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:41:20.426498 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:41:20.426788 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:41:20.426983 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0812 02:41:20.427314 (+ 331us) server_negotiation.cc:299] Negotiation successful
0812 02:41:20.427452 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260812 02:41:20.431979 29709 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54292:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 02:41:20.438977 29709 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 02:41:20.466465 30256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:20.452562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42472 (local address 127.7.132.130:36693)
0812 02:41:20.458019 (+ 5457us) server_negotiation.cc:206] Beginning negotiation
0812 02:41:20.458033 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 02:41:20.458067 (+ 34us) server_negotiation.cc:407] Connection header received
0812 02:41:20.458225 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:41:20.458233 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:41:20.458499 (+ 266us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:41:20.458780 (+ 281us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:41:20.459631 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:20.461253 (+ 1622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:20.463922 (+ 2669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:20.465300 (+ 1378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:20.465679 (+ 379us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:20.465883 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0812 02:41:20.465966 (+ 83us) server_negotiation.cc:299] Negotiation successful
0812 02:41:20.466165 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5249,"thread_start_us":5108,"threads_started":1}
I20260812 02:41:20.466765 30259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:20.454910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39486 (local address 127.7.132.131:36949)
0812 02:41:20.457649 (+ 2739us) server_negotiation.cc:206] Beginning negotiation
0812 02:41:20.457663 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 02:41:20.457703 (+ 40us) server_negotiation.cc:407] Connection header received
0812 02:41:20.457809 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:41:20.457817 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:41:20.458141 (+ 324us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:41:20.458416 (+ 275us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:41:20.459260 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:20.460714 (+ 1454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:20.463747 (+ 3033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:20.465194 (+ 1447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:20.465964 (+ 770us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:20.466208 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0812 02:41:20.466281 (+ 73us) server_negotiation.cc:299] Negotiation successful
0812 02:41:20.466425 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2509,"thread_start_us":2319,"threads_started":1}
I20260812 02:41:20.466809 30257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:20.452629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43300 (local address 127.7.132.129:38225)
0812 02:41:20.458443 (+ 5814us) server_negotiation.cc:206] Beginning negotiation
0812 02:41:20.458452 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 02:41:20.458481 (+ 29us) server_negotiation.cc:407] Connection header received
0812 02:41:20.458590 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:41:20.458598 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:41:20.458845 (+ 247us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:41:20.459455 (+ 610us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:41:20.460209 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:20.461636 (+ 1427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:20.463857 (+ 2221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:20.465096 (+ 1239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:20.465257 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:20.465511 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0812 02:41:20.466315 (+ 804us) server_negotiation.cc:299] Negotiation successful
0812 02:41:20.466486 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5627,"thread_start_us":5428,"threads_started":1}
I20260812 02:41:20.467185 30255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:20.451919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.130:36693 (local address 127.0.0.1:42472)
0812 02:41:20.453675 (+ 1756us) negotiation.cc:107] Waiting for socket to connect
0812 02:41:20.453745 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0812 02:41:20.453957 (+ 212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:41:20.458790 (+ 4833us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:41:20.458797 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:41:20.458844 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:41:20.459403 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:41:20.459418 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:41:20.461398 (+ 1980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:41:20.461406 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:41:20.463733 (+ 2327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:41:20.463746 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:41:20.465525 (+ 1779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:41:20.465534 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:41:20.465799 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:20.465827 (+ 28us) client_negotiation.cc:769] Sending connection context
0812 02:41:20.465897 (+ 70us) client_negotiation.cc:240] Negotiation successful
0812 02:41:20.466029 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1405,"mutex_wait_us":60,"thread_start_us":2420,"threads_started":1}
I20260812 02:41:20.467290 30254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:20.451993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:38225 (local address 127.0.0.1:43300)
0812 02:41:20.454639 (+ 2646us) negotiation.cc:107] Waiting for socket to connect
0812 02:41:20.454661 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 02:41:20.454795 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:41:20.459121 (+ 4326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:41:20.459127 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:41:20.459153 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:41:20.460002 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:41:20.460016 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:41:20.461777 (+ 1761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:41:20.461782 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:41:20.463652 (+ 1870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:41:20.463675 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:41:20.465309 (+ 1634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:41:20.465315 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:41:20.465553 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:20.465614 (+ 61us) client_negotiation.cc:769] Sending connection context
0812 02:41:20.465774 (+ 160us) client_negotiation.cc:240] Negotiation successful
0812 02:41:20.466036 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2507,"mutex_wait_us":473,"thread_start_us":1174,"threads_started":1}
I20260812 02:41:20.467509 30258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:20.454228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.131:36949 (local address 127.0.0.1:39486)
0812 02:41:20.456757 (+ 2529us) negotiation.cc:107] Waiting for socket to connect
0812 02:41:20.456785 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0812 02:41:20.456923 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:41:20.458443 (+ 1520us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:41:20.458451 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:41:20.458482 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:41:20.458992 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:41:20.459013 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:41:20.460853 (+ 1840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:41:20.460861 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:41:20.463526 (+ 2665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:41:20.463542 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:41:20.465461 (+ 1919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:41:20.465468 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:41:20.465697 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:20.465727 (+ 30us) client_negotiation.cc:769] Sending connection context
0812 02:41:20.465798 (+ 71us) client_negotiation.cc:240] Negotiation successful
0812 02:41:20.466213 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2377,"mutex_wait_us":362,"thread_start_us":2218,"threads_started":1}
I20260812 02:41:20.471074 30003 tablet_service.cc:1467] Processing CreateTablet for tablet 0dbbb1ed9804456f9e110be0be77eec5 (DEFAULT_TABLE table=test-table [id=2203ae95c7ca43fab956b1ced57b4e56]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:41:20.471243 30165 tablet_service.cc:1467] Processing CreateTablet for tablet 0dbbb1ed9804456f9e110be0be77eec5 (DEFAULT_TABLE table=test-table [id=2203ae95c7ca43fab956b1ced57b4e56]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:41:20.471243 29848 tablet_service.cc:1467] Processing CreateTablet for tablet 0dbbb1ed9804456f9e110be0be77eec5 (DEFAULT_TABLE table=test-table [id=2203ae95c7ca43fab956b1ced57b4e56]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:41:20.472530 30003 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0dbbb1ed9804456f9e110be0be77eec5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:41:20.472661 29848 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0dbbb1ed9804456f9e110be0be77eec5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:41:20.472661 30165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0dbbb1ed9804456f9e110be0be77eec5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:41:20.484812 30260 tablet_bootstrap.cc:492] T 0dbbb1ed9804456f9e110be0be77eec5 P 7d88fea233a24b47870378b942d168ea: Bootstrap starting.
I20260812 02:41:20.487771 30261 tablet_bootstrap.cc:492] T 0dbbb1ed9804456f9e110be0be77eec5 P edf58a468a2843b393738669a59ae39f: Bootstrap starting.
I20260812 02:41:20.488956 30263 tablet_bootstrap.cc:492] T 0dbbb1ed9804456f9e110be0be77eec5 P 081da6508f944603b63ee1f8472e7f73: Bootstrap starting.
I20260812 02:41:20.489231 30260 tablet_bootstrap.cc:654] T 0dbbb1ed9804456f9e110be0be77eec5 P 7d88fea233a24b47870378b942d168ea: Neither blocks nor log segments found. Creating new log.
I20260812 02:41:20.490736 30260 log.cc:824] T 0dbbb1ed9804456f9e110be0be77eec5 P 7d88fea233a24b47870378b942d168ea: Log is configured to *not* fsync() on all Append() calls
I20260812 02:41:20.491791 30261 tablet_bootstrap.cc:654] T 0dbbb1ed9804456f9e110be0be77eec5 P edf58a468a2843b393738669a59ae39f: Neither blocks nor log segments found. Creating new log.
I20260812 02:41:20.492471 30263 tablet_bootstrap.cc:654] T 0dbbb1ed9804456f9e110be0be77eec5 P 081da6508f944603b63ee1f8472e7f73: Neither blocks nor log segments found. Creating new log.
I20260812 02:41:20.493490 30263 log.cc:824] T 0dbbb1ed9804456f9e110be0be77eec5 P 081da6508f944603b63ee1f8472e7f73: Log is configured to *not* fsync() on all Append() calls
I20260812 02:41:20.493466 30261 log.cc:824] T 0dbbb1ed9804456f9e110be0be77eec5 P edf58a468a2843b393738669a59ae39f: Log is configured to *not* fsync() on all Append() calls
I20260812 02:41:20.496498 30263 tablet_bootstrap.cc:492] T 0dbbb1ed9804456f9e110be0be77eec5 P 081da6508f944603b63ee1f8472e7f73: No bootstrap required, opened a new log
I20260812 02:41:20.496769 30261 tablet_bootstrap.cc:492] T 0dbbb1ed9804456f9e110be0be77eec5 P edf58a468a2843b393738669a59ae39f: No bootstrap required, opened a new log
I20260812 02:41:20.496771 30260 tablet_bootstrap.cc:492] T 0dbbb1ed9804456f9e110be0be77eec5 P 7d88fea233a24b47870378b942d168ea: No bootstrap required, opened a new log
I20260812 02:41:20.496819 30263 ts_tablet_manager.cc:1397] T 0dbbb1ed9804456f9e110be0be77eec5 P 081da6508f944603b63ee1f8472e7f73: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260812 02:41:20.497057 30260 ts_tablet_manager.cc:1397] T 0dbbb1ed9804456f9e110be0be77eec5 P 7d88fea233a24b47870378b942d168ea: Time spent bootstrapping tablet: real 0.012s user 0.009s sys 0.000s
I20260812 02:41:20.497057 30261 ts_tablet_manager.cc:1397] T 0dbbb1ed9804456f9e110be0be77eec5 P edf58a468a2843b393738669a59ae39f: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.001s
I20260812 02:41:20.504645 30263 raft_consensus.cc:348] T 0dbbb1ed9804456f9e110be0be77eec5 P 081da6508f944603b63ee1f8472e7f73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d88fea233a24b47870378b942d168ea" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 36693 } } peers { permanent_uuid: "edf58a468a2843b393738669a59ae39f" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 38225 } } peers { permanent_uuid: "081da6508f944603b63ee1f8472e7f73" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 36949 } }
I20260812 02:41:20.505091 30263 raft_consensus.cc:374] T 0dbbb1ed9804456f9e110be0be77eec5 P 081da6508f944603b63ee1f8472e7f73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:41:20.505198 30263 raft_consensus.cc:729] T 0dbbb1ed9804456f9e110be0be77eec5 P 081da6508f944603b63ee1f8472e7f73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 081da6508f944603b63ee1f8472e7f73, State: Initialized, Role: FOLLOWER
I20260812 02:41:20.505887 30263 consensus_queue.cc:260] T 0dbbb1ed9804456f9e110be0be77eec5 P 081da6508f944603b63ee1f8472e7f73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d88fea233a24b47870378b942d168ea" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 36693 } } peers { permanent_uuid: "edf58a468a2843b393738669a59ae39f" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 38225 } } peers { permanent_uuid: "081da6508f944603b63ee1f8472e7f73" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 36949 } }
I20260812 02:41:20.507190 30240 heartbeater.cc:503] Master 127.7.132.190:38321 was elected leader, sending a full tablet report...
I20260812 02:41:20.507838 30263 ts_tablet_manager.cc:1428] T 0dbbb1ed9804456f9e110be0be77eec5 P 081da6508f944603b63ee1f8472e7f73: Time spent starting tablet: real 0.011s user 0.011s sys 0.000s
I20260812 02:41:20.508162 30260 raft_consensus.cc:348] T 0dbbb1ed9804456f9e110be0be77eec5 P 7d88fea233a24b47870378b942d168ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d88fea233a24b47870378b942d168ea" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 36693 } } peers { permanent_uuid: "edf58a468a2843b393738669a59ae39f" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 38225 } } peers { permanent_uuid: "081da6508f944603b63ee1f8472e7f73" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 36949 } }
I20260812 02:41:20.508729 30260 raft_consensus.cc:374] T 0dbbb1ed9804456f9e110be0be77eec5 P 7d88fea233a24b47870378b942d168ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:41:20.508879 30260 raft_consensus.cc:729] T 0dbbb1ed9804456f9e110be0be77eec5 P 7d88fea233a24b47870378b942d168ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d88fea233a24b47870378b942d168ea, State: Initialized, Role: FOLLOWER
I20260812 02:41:20.509043 30261 raft_consensus.cc:348] T 0dbbb1ed9804456f9e110be0be77eec5 P edf58a468a2843b393738669a59ae39f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d88fea233a24b47870378b942d168ea" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 36693 } } peers { permanent_uuid: "edf58a468a2843b393738669a59ae39f" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 38225 } } peers { permanent_uuid: "081da6508f944603b63ee1f8472e7f73" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 36949 } }
I20260812 02:41:20.509454 30261 raft_consensus.cc:374] T 0dbbb1ed9804456f9e110be0be77eec5 P edf58a468a2843b393738669a59ae39f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:41:20.509526 30261 raft_consensus.cc:729] T 0dbbb1ed9804456f9e110be0be77eec5 P edf58a468a2843b393738669a59ae39f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: edf58a468a2843b393738669a59ae39f, State: Initialized, Role: FOLLOWER
I20260812 02:41:20.509553 30260 consensus_queue.cc:260] T 0dbbb1ed9804456f9e110be0be77eec5 P 7d88fea233a24b47870378b942d168ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d88fea233a24b47870378b942d168ea" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 36693 } } peers { permanent_uuid: "edf58a468a2843b393738669a59ae39f" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 38225 } } peers { permanent_uuid: "081da6508f944603b63ee1f8472e7f73" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 36949 } }
I20260812 02:41:20.510064 30261 consensus_queue.cc:260] T 0dbbb1ed9804456f9e110be0be77eec5 P edf58a468a2843b393738669a59ae39f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d88fea233a24b47870378b942d168ea" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 36693 } } peers { permanent_uuid: "edf58a468a2843b393738669a59ae39f" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 38225 } } peers { permanent_uuid: "081da6508f944603b63ee1f8472e7f73" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 36949 } }
I20260812 02:41:20.511443 30260 ts_tablet_manager.cc:1428] T 0dbbb1ed9804456f9e110be0be77eec5 P 7d88fea233a24b47870378b942d168ea: Time spent starting tablet: real 0.014s user 0.007s sys 0.008s
I20260812 02:41:20.512069 30082 heartbeater.cc:503] Master 127.7.132.190:38321 was elected leader, sending a full tablet report...
I20260812 02:41:20.515552 30261 ts_tablet_manager.cc:1428] T 0dbbb1ed9804456f9e110be0be77eec5 P edf58a468a2843b393738669a59ae39f: Time spent starting tablet: real 0.018s user 0.012s sys 0.006s
W20260812 02:41:20.522815 29923 tablet.cc:2369] T 0dbbb1ed9804456f9e110be0be77eec5 P edf58a468a2843b393738669a59ae39f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 02:41:20.587446 30083 tablet.cc:2369] T 0dbbb1ed9804456f9e110be0be77eec5 P 7d88fea233a24b47870378b942d168ea: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 02:41:20.630963 30241 tablet.cc:2369] T 0dbbb1ed9804456f9e110be0be77eec5 P 081da6508f944603b63ee1f8472e7f73: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 02:41:20.836243 30267 raft_consensus.cc:482] T 0dbbb1ed9804456f9e110be0be77eec5 P 081da6508f944603b63ee1f8472e7f73 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 02:41:20.836586 30267 raft_consensus.cc:504] T 0dbbb1ed9804456f9e110be0be77eec5 P 081da6508f944603b63ee1f8472e7f73 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d88fea233a24b47870378b942d168ea" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 36693 } } peers { permanent_uuid: "edf58a468a2843b393738669a59ae39f" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 38225 } } peers { permanent_uuid: "081da6508f944603b63ee1f8472e7f73" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 36949 } }
I20260812 02:41:20.838660 30267 leader_election.cc:288] T 0dbbb1ed9804456f9e110be0be77eec5 P 081da6508f944603b63ee1f8472e7f73 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7d88fea233a24b47870378b942d168ea (127.7.132.130:36693), edf58a468a2843b393738669a59ae39f (127.7.132.129:38225)
I20260812 02:41:20.846025 30256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:20.839516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:56433 (local address 127.7.132.130:36693)
0812 02:41:20.839806 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0812 02:41:20.839811 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:41:20.839829 (+ 18us) server_negotiation.cc:407] Connection header received
0812 02:41:20.839891 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:41:20.839896 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:41:20.839964 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:41:20.840042 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:41:20.840575 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:20.841340 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:20.843879 (+ 2539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:20.845122 (+ 1243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:20.845305 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:20.845380 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 02:41:20.845636 (+ 256us) server_negotiation.cc:299] Negotiation successful
0812 02:41:20.845756 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135}
I20260812 02:41:20.846019 30243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:20.838944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.130:36693 (local address 127.7.132.131:56433)
0812 02:41:20.839425 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0812 02:41:20.839444 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 02:41:20.839567 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:41:20.840207 (+ 640us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:41:20.840213 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:41:20.840237 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:41:20.840421 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:41:20.840431 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:41:20.841491 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:41:20.841497 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:41:20.843691 (+ 2194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:41:20.843706 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:41:20.845280 (+ 1574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:41:20.845285 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:41:20.845468 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:20.845493 (+ 25us) client_negotiation.cc:769] Sending connection context
0812 02:41:20.845597 (+ 104us) client_negotiation.cc:240] Negotiation successful
0812 02:41:20.845736 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":367,"mutex_wait_us":54}
I20260812 02:41:20.847208 30257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:20.839064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:55429 (local address 127.7.132.129:38225)
0812 02:41:20.839285 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0812 02:41:20.839292 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:41:20.840146 (+ 854us) server_negotiation.cc:407] Connection header received
0812 02:41:20.840371 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:41:20.840375 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:41:20.840436 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:41:20.840539 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:41:20.842070 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:20.842801 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:20.844963 (+ 2162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:20.846202 (+ 1239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:20.846348 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:20.846416 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 02:41:20.846828 (+ 412us) server_negotiation.cc:299] Negotiation successful
0812 02:41:20.846960 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260812 02:41:20.847329 30271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:20.839032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:38225 (local address 127.7.132.131:55429)
0812 02:41:20.840007 (+ 975us) negotiation.cc:107] Waiting for socket to connect
0812 02:41:20.840027 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 02:41:20.840197 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:41:20.841483 (+ 1286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:41:20.841487 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:41:20.841507 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:41:20.841854 (+ 347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:41:20.841863 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:41:20.842957 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:41:20.842968 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:41:20.844753 (+ 1785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:41:20.844769 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:41:20.846442 (+ 1673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:41:20.846449 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:41:20.846656 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:20.846693 (+ 37us) client_negotiation.cc:769] Sending connection context
0812 02:41:20.846794 (+ 101us) client_negotiation.cc:240] Negotiation successful
0812 02:41:20.846948 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":858,"mutex_wait_us":1,"thread_start_us":701,"threads_started":1}
I20260812 02:41:20.847348 30027 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0dbbb1ed9804456f9e110be0be77eec5" candidate_uuid: "081da6508f944603b63ee1f8472e7f73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d88fea233a24b47870378b942d168ea" is_pre_election: true
I20260812 02:41:20.847914 30027 raft_consensus.cc:2454] T 0dbbb1ed9804456f9e110be0be77eec5 P 7d88fea233a24b47870378b942d168ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 081da6508f944603b63ee1f8472e7f73 in term 0.
I20260812 02:41:20.848619 30111 leader_election.cc:302] T 0dbbb1ed9804456f9e110be0be77eec5 P 081da6508f944603b63ee1f8472e7f73 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 081da6508f944603b63ee1f8472e7f73, 7d88fea233a24b47870378b942d168ea; no voters:
I20260812 02:41:20.848567 29872 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0dbbb1ed9804456f9e110be0be77eec5" candidate_uuid: "081da6508f944603b63ee1f8472e7f73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "edf58a468a2843b393738669a59ae39f" is_pre_election: true
I20260812 02:41:20.849012 30267 raft_consensus.cc:2781] T 0dbbb1ed9804456f9e110be0be77eec5 P 081da6508f944603b63ee1f8472e7f73 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 02:41:20.849076 29872 raft_consensus.cc:2454] T 0dbbb1ed9804456f9e110be0be77eec5 P edf58a468a2843b393738669a59ae39f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 081da6508f944603b63ee1f8472e7f73 in term 0.
I20260812 02:41:20.849181 30267 raft_consensus.cc:482] T 0dbbb1ed9804456f9e110be0be77eec5 P 081da6508f944603b63ee1f8472e7f73 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 02:41:20.849283 30267 raft_consensus.cc:3037] T 0dbbb1ed9804456f9e110be0be77eec5 P 081da6508f944603b63ee1f8472e7f73 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:41:20.851585 30267 raft_consensus.cc:504] T 0dbbb1ed9804456f9e110be0be77eec5 P 081da6508f944603b63ee1f8472e7f73 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d88fea233a24b47870378b942d168ea" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 36693 } } peers { permanent_uuid: "edf58a468a2843b393738669a59ae39f" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 38225 } } peers { permanent_uuid: "081da6508f944603b63ee1f8472e7f73" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 36949 } }
I20260812 02:41:20.852108 30267 leader_election.cc:288] T 0dbbb1ed9804456f9e110be0be77eec5 P 081da6508f944603b63ee1f8472e7f73 [CANDIDATE]: Term 1 election: Requested vote from peers 7d88fea233a24b47870378b942d168ea (127.7.132.130:36693), edf58a468a2843b393738669a59ae39f (127.7.132.129:38225)
I20260812 02:41:20.852531 30027 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0dbbb1ed9804456f9e110be0be77eec5" candidate_uuid: "081da6508f944603b63ee1f8472e7f73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d88fea233a24b47870378b942d168ea"
I20260812 02:41:20.852571 29872 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0dbbb1ed9804456f9e110be0be77eec5" candidate_uuid: "081da6508f944603b63ee1f8472e7f73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "edf58a468a2843b393738669a59ae39f"
I20260812 02:41:20.852715 30027 raft_consensus.cc:3037] T 0dbbb1ed9804456f9e110be0be77eec5 P 7d88fea233a24b47870378b942d168ea [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:41:20.852818 29872 raft_consensus.cc:3037] T 0dbbb1ed9804456f9e110be0be77eec5 P edf58a468a2843b393738669a59ae39f [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:41:20.855006 30027 raft_consensus.cc:2454] T 0dbbb1ed9804456f9e110be0be77eec5 P 7d88fea233a24b47870378b942d168ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 081da6508f944603b63ee1f8472e7f73 in term 1.
I20260812 02:41:20.855176 29872 raft_consensus.cc:2454] T 0dbbb1ed9804456f9e110be0be77eec5 P edf58a468a2843b393738669a59ae39f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 081da6508f944603b63ee1f8472e7f73 in term 1.
I20260812 02:41:20.855435 30111 leader_election.cc:302] T 0dbbb1ed9804456f9e110be0be77eec5 P 081da6508f944603b63ee1f8472e7f73 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 081da6508f944603b63ee1f8472e7f73, 7d88fea233a24b47870378b942d168ea; no voters:
I20260812 02:41:20.855710 30267 raft_consensus.cc:2781] T 0dbbb1ed9804456f9e110be0be77eec5 P 081da6508f944603b63ee1f8472e7f73 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:41:20.856133 30267 raft_consensus.cc:686] T 0dbbb1ed9804456f9e110be0be77eec5 P 081da6508f944603b63ee1f8472e7f73 [term 1 LEADER]: Becoming Leader. State: Replica: 081da6508f944603b63ee1f8472e7f73, State: Running, Role: LEADER
I20260812 02:41:20.856580 30267 consensus_queue.cc:237] T 0dbbb1ed9804456f9e110be0be77eec5 P 081da6508f944603b63ee1f8472e7f73 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d88fea233a24b47870378b942d168ea" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 36693 } } peers { permanent_uuid: "edf58a468a2843b393738669a59ae39f" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 38225 } } peers { permanent_uuid: "081da6508f944603b63ee1f8472e7f73" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 36949 } }
I20260812 02:41:20.861469 29708 catalog_manager.cc:5495] T 0dbbb1ed9804456f9e110be0be77eec5 P 081da6508f944603b63ee1f8472e7f73 reported cstate change: term changed from 0 to 1, leader changed from <none> to 081da6508f944603b63ee1f8472e7f73 (127.7.132.131). New cstate: current_term: 1 leader_uuid: "081da6508f944603b63ee1f8472e7f73" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d88fea233a24b47870378b942d168ea" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 36693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "edf58a468a2843b393738669a59ae39f" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 38225 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "081da6508f944603b63ee1f8472e7f73" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 36949 } health_report { overall_health: HEALTHY } } }
I20260812 02:41:20.918979 30259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:20.913441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39500 (local address 127.7.132.131:36949)
0812 02:41:20.913676 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0812 02:41:20.913681 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:41:20.913699 (+ 18us) server_negotiation.cc:407] Connection header received
0812 02:41:20.913893 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:41:20.913898 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:41:20.913979 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:41:20.914200 (+ 221us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 02:41:20.914604 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:20.915314 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:20.916713 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:20.917095 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:20.917191 (+ 96us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:20.917748 (+ 557us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 02:41:20.918325 (+ 577us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 02:41:20.918464 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0812 02:41:20.918622 (+ 158us) server_negotiation.cc:299] Negotiation successful
0812 02:41:20.918692 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260812 02:41:20.927371 29872 raft_consensus.cc:1261] T 0dbbb1ed9804456f9e110be0be77eec5 P edf58a468a2843b393738669a59ae39f [term 1 FOLLOWER]: Refusing update from remote peer 081da6508f944603b63ee1f8472e7f73: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:41:20.927372 30027 raft_consensus.cc:1261] T 0dbbb1ed9804456f9e110be0be77eec5 P 7d88fea233a24b47870378b942d168ea [term 1 FOLLOWER]: Refusing update from remote peer 081da6508f944603b63ee1f8472e7f73: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:41:20.928347 30267 consensus_queue.cc:1035] T 0dbbb1ed9804456f9e110be0be77eec5 P 081da6508f944603b63ee1f8472e7f73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "edf58a468a2843b393738669a59ae39f" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 38225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:41:20.928692 30272 consensus_queue.cc:1035] T 0dbbb1ed9804456f9e110be0be77eec5 P 081da6508f944603b63ee1f8472e7f73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d88fea233a24b47870378b942d168ea" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 36693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:41:20.940891 30279 mvcc.cc:205] Tried to move back new op lower bound from 7317514161868267520 to 7317514161592487936. Current Snapshot: MvccSnapshot[applied={T|T < 7317514161868267520}]
I20260812 02:41:20.943670 30275 mvcc.cc:205] Tried to move back new op lower bound from 7317514161868267520 to 7317514161592487936. Current Snapshot: MvccSnapshot[applied={T|T < 7317514161868267520}]
I20260812 02:41:20.957511 29709 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54292:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 02:41:20.957790 29709 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54292:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 02:41:20.961339 29709 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f9442c6a8c994b7dba3179b01e1dfcd0: Sending DeleteTablet for 3 replicas of tablet 0dbbb1ed9804456f9e110be0be77eec5
I20260812 02:41:20.962492 30003 tablet_service.cc:1514] Processing DeleteTablet for tablet 0dbbb1ed9804456f9e110be0be77eec5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:41:20 UTC) from {username='kudu', principal='kudu/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:42472
I20260812 02:41:20.962661 29848 tablet_service.cc:1514] Processing DeleteTablet for tablet 0dbbb1ed9804456f9e110be0be77eec5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:41:20 UTC) from {username='kudu', principal='kudu/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:43300
I20260812 02:41:20.963819 30165 tablet_service.cc:1514] Processing DeleteTablet for tablet 0dbbb1ed9804456f9e110be0be77eec5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:41:20 UTC) from {username='kudu', principal='kudu/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:39486
I20260812 02:41:20.964582 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 29778
I20260812 02:41:20.974239 30289 tablet_replica.cc:330] T 0dbbb1ed9804456f9e110be0be77eec5 P edf58a468a2843b393738669a59ae39f: stopping tablet replica
I20260812 02:41:20.974871 30289 raft_consensus.cc:2229] T 0dbbb1ed9804456f9e110be0be77eec5 P edf58a468a2843b393738669a59ae39f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 02:41:20.975513 30289 raft_consensus.cc:2258] T 0dbbb1ed9804456f9e110be0be77eec5 P edf58a468a2843b393738669a59ae39f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:41:20.978376 30289 ts_tablet_manager.cc:1905] T 0dbbb1ed9804456f9e110be0be77eec5 P edf58a468a2843b393738669a59ae39f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:41:20.983618 30290 tablet_replica.cc:330] T 0dbbb1ed9804456f9e110be0be77eec5 P 081da6508f944603b63ee1f8472e7f73: stopping tablet replica
I20260812 02:41:20.984272 30290 raft_consensus.cc:2229] T 0dbbb1ed9804456f9e110be0be77eec5 P 081da6508f944603b63ee1f8472e7f73 [term 1 LEADER]: Raft consensus shutting down.
I20260812 02:41:20.985255 30290 raft_consensus.cc:2258] T 0dbbb1ed9804456f9e110be0be77eec5 P 081da6508f944603b63ee1f8472e7f73 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:41:20.986508 30289 ts_tablet_manager.cc:1918] T 0dbbb1ed9804456f9e110be0be77eec5 P edf58a468a2843b393738669a59ae39f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 02:41:20.986817 30289 log.cc:1196] T 0dbbb1ed9804456f9e110be0be77eec5 P edf58a468a2843b393738669a59ae39f: Deleting WAL directory at /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502156971023-7698-0/minicluster-data/ts-0/wal/wals/0dbbb1ed9804456f9e110be0be77eec5
I20260812 02:41:20.987511 30289 ts_tablet_manager.cc:1939] T 0dbbb1ed9804456f9e110be0be77eec5 P edf58a468a2843b393738669a59ae39f: Deleting consensus metadata
I20260812 02:41:20.989717 29692 catalog_manager.cc:4841] TS edf58a468a2843b393738669a59ae39f (127.7.132.129:38225): tablet 0dbbb1ed9804456f9e110be0be77eec5 (table test-table [id=2203ae95c7ca43fab956b1ced57b4e56]) successfully deleted
I20260812 02:41:20.990751 30290 ts_tablet_manager.cc:1905] T 0dbbb1ed9804456f9e110be0be77eec5 P 081da6508f944603b63ee1f8472e7f73: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:41:20.994011 30288 tablet_replica.cc:330] T 0dbbb1ed9804456f9e110be0be77eec5 P 7d88fea233a24b47870378b942d168ea: stopping tablet replica
I20260812 02:41:20.994558 30288 raft_consensus.cc:2229] T 0dbbb1ed9804456f9e110be0be77eec5 P 7d88fea233a24b47870378b942d168ea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 02:41:20.995198 30288 raft_consensus.cc:2258] T 0dbbb1ed9804456f9e110be0be77eec5 P 7d88fea233a24b47870378b942d168ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:41:20.997488 30288 ts_tablet_manager.cc:1905] T 0dbbb1ed9804456f9e110be0be77eec5 P 7d88fea233a24b47870378b942d168ea: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 12 02:41:20 dist-test-slave-mn4r krb5kdc[29661](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502478, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.132.129@KRBTEST.COM
I20260812 02:41:20.999540 30290 ts_tablet_manager.cc:1918] T 0dbbb1ed9804456f9e110be0be77eec5 P 081da6508f944603b63ee1f8472e7f73: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 02:41:20.999745 30290 log.cc:1196] T 0dbbb1ed9804456f9e110be0be77eec5 P 081da6508f944603b63ee1f8472e7f73: Deleting WAL directory at /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502156971023-7698-0/minicluster-data/ts-2/wal/wals/0dbbb1ed9804456f9e110be0be77eec5
I20260812 02:41:21.000387 30290 ts_tablet_manager.cc:1939] T 0dbbb1ed9804456f9e110be0be77eec5 P 081da6508f944603b63ee1f8472e7f73: Deleting consensus metadata
I20260812 02:41:21.002404 29692 catalog_manager.cc:4841] TS 081da6508f944603b63ee1f8472e7f73 (127.7.132.131:36949): tablet 0dbbb1ed9804456f9e110be0be77eec5 (table test-table [id=2203ae95c7ca43fab956b1ced57b4e56]) successfully deleted
I20260812 02:41:21.005376 30288 ts_tablet_manager.cc:1918] T 0dbbb1ed9804456f9e110be0be77eec5 P 7d88fea233a24b47870378b942d168ea: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 02:41:21.005642 30288 log.cc:1196] T 0dbbb1ed9804456f9e110be0be77eec5 P 7d88fea233a24b47870378b942d168ea: Deleting WAL directory at /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786502156971023-7698-0/minicluster-data/ts-1/wal/wals/0dbbb1ed9804456f9e110be0be77eec5
I20260812 02:41:21.006179 30257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:20.965914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43312 (local address 127.7.132.129:38225)
0812 02:41:20.966163 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0812 02:41:20.966172 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 02:41:20.971356 (+ 5184us) server_negotiation.cc:407] Connection header received
0812 02:41:20.971604 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:41:20.971613 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:41:20.971758 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:41:20.971872 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:41:20.985863 (+ 13991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:20.987438 (+ 1575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:20.991873 (+ 4435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:20.992537 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:20.992659 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:20.998954 (+ 6295us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:41:20.998972 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:41:20.998989 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:41:20.999047 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:41:21.003081 (+ 4034us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:41:21.003881 (+ 800us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:41:21.003890 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:41:21.003892 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:41:21.004041 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:41:21.004461 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:41:21.004465 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:41:21.004467 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:41:21.004826 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:41:21.005169 (+ 343us) server_negotiation.cc:1035] Waiting for connection context
0812 02:41:21.005500 (+ 331us) server_negotiation.cc:299] Negotiation successful
0812 02:41:21.005747 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260812 02:41:21.006340 30288 ts_tablet_manager.cc:1939] T 0dbbb1ed9804456f9e110be0be77eec5 P 7d88fea233a24b47870378b942d168ea: Deleting consensus metadata
I20260812 02:41:21.006912 29917 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:41:21.008250 29694 catalog_manager.cc:4841] TS 7d88fea233a24b47870378b942d168ea (127.7.132.130:36693): tablet 0dbbb1ed9804456f9e110be0be77eec5 (table test-table [id=2203ae95c7ca43fab956b1ced57b4e56]) successfully deleted
I20260812 02:41:21.100179 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 29778
I20260812 02:41:21.125077 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 29928
Aug 12 02:41:21 dist-test-slave-mn4r krb5kdc[29661](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502478, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.132.130@KRBTEST.COM
I20260812 02:41:21.142472 30256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:21.125940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42484 (local address 127.7.132.130:36693)
0812 02:41:21.126151 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0812 02:41:21.126157 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:41:21.126184 (+ 27us) server_negotiation.cc:407] Connection header received
0812 02:41:21.126376 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:41:21.126383 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:41:21.126475 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:41:21.126591 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:41:21.127842 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:21.128954 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:21.130455 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:21.131109 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:21.131237 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:21.135819 (+ 4582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:41:21.135841 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:41:21.135862 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:41:21.135908 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:41:21.139597 (+ 3689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:41:21.140273 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:41:21.140283 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:41:21.140287 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:41:21.140458 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:41:21.140814 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:41:21.140820 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:41:21.140825 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:41:21.141291 (+ 466us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:41:21.141627 (+ 336us) server_negotiation.cc:1035] Waiting for connection context
0812 02:41:21.141803 (+ 176us) server_negotiation.cc:299] Negotiation successful
0812 02:41:21.142052 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260812 02:41:21.143224 30077 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:41:21.249807 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 29928
I20260812 02:41:21.280783 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 30090
Aug 12 02:41:21 dist-test-slave-mn4r krb5kdc[29661](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502478, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.132.131@KRBTEST.COM
I20260812 02:41:21.298679 30259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:21.281713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39514 (local address 127.7.132.131:36949)
0812 02:41:21.281963 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0812 02:41:21.281970 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:41:21.281991 (+ 21us) server_negotiation.cc:407] Connection header received
0812 02:41:21.282188 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:41:21.282192 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:41:21.282269 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:41:21.282381 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:41:21.283836 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:21.284724 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:21.286020 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:21.286606 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:21.286762 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:21.292024 (+ 5262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:41:21.292038 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:41:21.292054 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:41:21.292093 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:41:21.295494 (+ 3401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:41:21.296398 (+ 904us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:41:21.296406 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:41:21.296408 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:41:21.296578 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:41:21.297053 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:41:21.297056 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:41:21.297058 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:41:21.297432 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:41:21.297664 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0812 02:41:21.298143 (+ 479us) server_negotiation.cc:299] Negotiation successful
0812 02:41:21.298333 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
I20260812 02:41:21.299441 30233 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:41:21.428689 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 30090
I20260812 02:41:21.453410 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 29676
I20260812 02:41:21.454667 29744 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:41:21.568528 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 29676
2026-08-12T02:41:21Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (5028 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-12T02:41:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T02:41:21Z Disabled control of system clock
I20260812 02:41:21.663007 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502156971023-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502156971023-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502156971023-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:37605
--webserver_interface=127.7.132.190
--webserver_port=0
--builtin_ntp_servers=127.7.132.148:39375
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.132.190:37605
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 02:41:22.011986 30300 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:41:22.012286 30300 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:41:22.012368 30300 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:41:22.019502 30300 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:41:22.019614 30300 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:41:22.019662 30300 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:41:22.019727 30300 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:41:22.019769 30300 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:41:22.028847 30300 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:39375
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502156971023-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502156971023-7698-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.132.190:37605
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:37605
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.132.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502156971023-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:41:22.030121 30300 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:41:22.032023 30300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:41:22.044227 30308 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:41:22.044275 30311 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:41:22.044334 30309 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:41:22.045189 30300 server_base.cc:1034] running on GCE node
I20260812 02:41:22.045931 30300 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:41:22.047492 30300 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:41:22.048700 30300 hybrid_clock.cc:648] HybridClock initialized: now 1786502482048656 us; error 30 us; skew 500 ppm
I20260812 02:41:22.049082 30300 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 02:41:22.051527 30300 webserver.cc:460] Webserver started at http://127.7.132.190:37723/ using document root <none> and password file <none>
I20260812 02:41:22.052279 30300 fs_manager.cc:359] Metadata directory not provided
I20260812 02:41:22.052426 30300 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:41:22.052783 30300 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:41:22.055812 30300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502156971023-7698-0/minicluster-data/master-0/data/instance:
uuid: "a2e16a9e23b24d8fa7fb4b0ee6452e5e"
format_stamp: "Formatted at 2026-08-12 02:41:22 on dist-test-slave-mn4r"
server_key: "f1f833ae8efb8ec51dbe6c1e0b99a2a8"
server_key_iv: "772c3d8e9be19c0381fbb455de6152ef"
server_key_version: "encryptionkey@0"
I20260812 02:41:22.056640 30300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502156971023-7698-0/minicluster-data/master-0/wal/instance:
uuid: "a2e16a9e23b24d8fa7fb4b0ee6452e5e"
format_stamp: "Formatted at 2026-08-12 02:41:22 on dist-test-slave-mn4r"
server_key: "f1f833ae8efb8ec51dbe6c1e0b99a2a8"
server_key_iv: "772c3d8e9be19c0381fbb455de6152ef"
server_key_version: "encryptionkey@0"
I20260812 02:41:22.061820 30300 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260812 02:41:22.065382 30318 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:41:22.066824 30300 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 02:41:22.067021 30300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502156971023-7698-0/minicluster-data/master-0/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502156971023-7698-0/minicluster-data/master-0/wal
uuid: "a2e16a9e23b24d8fa7fb4b0ee6452e5e"
format_stamp: "Formatted at 2026-08-12 02:41:22 on dist-test-slave-mn4r"
server_key: "f1f833ae8efb8ec51dbe6c1e0b99a2a8"
server_key_iv: "772c3d8e9be19c0381fbb455de6152ef"
server_key_version: "encryptionkey@0"
I20260812 02:41:22.067257 30300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502156971023-7698-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502156971023-7698-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502156971023-7698-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:41:22.084470 30300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:41:22.085364 30300 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:41:22.085670 30300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:41:22.104851 30300 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.190:37605
I20260812 02:41:22.104863 30378 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.190:37605 every 8 connection(s)
I20260812 02:41:22.106339 30300 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
I20260812 02:41:22.110369 30379 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:41:22.112879 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 30300
I20260812 02:41:22.113067 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502156971023-7698-0/minicluster-data/master-0/wal/instance
I20260812 02:41:22.113510 7698 external_mini_cluster.cc:1428] Setting key dbd21984a4d1a4ef3794463421b38882
I20260812 02:41:22.122809 30379 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a2e16a9e23b24d8fa7fb4b0ee6452e5e: Bootstrap starting.
I20260812 02:41:22.126848 30379 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a2e16a9e23b24d8fa7fb4b0ee6452e5e: Neither blocks nor log segments found. Creating new log.
I20260812 02:41:22.128077 30379 log.cc:824] T 00000000000000000000000000000000 P a2e16a9e23b24d8fa7fb4b0ee6452e5e: Log is configured to *not* fsync() on all Append() calls
I20260812 02:41:22.128630 30383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:22.115579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40592 (local address 127.7.132.190:37605)
0812 02:41:22.116683 (+ 1104us) server_negotiation.cc:206] Beginning negotiation
0812 02:41:22.116694 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 02:41:22.116739 (+ 45us) server_negotiation.cc:407] Connection header received
0812 02:41:22.118558 (+ 1819us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:41:22.118582 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:41:22.119074 (+ 492us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:41:22.119426 (+ 352us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:41:22.120086 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:22.121387 (+ 1301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:22.123405 (+ 2018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:22.124132 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:22.124279 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:22.125672 (+ 1393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:41:22.125681 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:41:22.125695 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:41:22.126587 (+ 892us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:41:22.126692 (+ 105us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:41:22.126720 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:41:22.126836 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0812 02:41:22.127121 (+ 285us) server_negotiation.cc:299] Negotiation successful
0812 02:41:22.127502 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":692,"thread_start_us":428,"threads_started":1}
I20260812 02:41:22.131286 30379 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a2e16a9e23b24d8fa7fb4b0ee6452e5e: No bootstrap required, opened a new log
I20260812 02:41:22.139071 30379 raft_consensus.cc:348] T 00000000000000000000000000000000 P a2e16a9e23b24d8fa7fb4b0ee6452e5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2e16a9e23b24d8fa7fb4b0ee6452e5e" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 37605 } }
I20260812 02:41:22.139424 30379 raft_consensus.cc:374] T 00000000000000000000000000000000 P a2e16a9e23b24d8fa7fb4b0ee6452e5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:41:22.139503 30379 raft_consensus.cc:729] T 00000000000000000000000000000000 P a2e16a9e23b24d8fa7fb4b0ee6452e5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2e16a9e23b24d8fa7fb4b0ee6452e5e, State: Initialized, Role: FOLLOWER
I20260812 02:41:22.140004 30379 consensus_queue.cc:260] T 00000000000000000000000000000000 P a2e16a9e23b24d8fa7fb4b0ee6452e5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2e16a9e23b24d8fa7fb4b0ee6452e5e" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 37605 } }
I20260812 02:41:22.140252 30379 raft_consensus.cc:388] T 00000000000000000000000000000000 P a2e16a9e23b24d8fa7fb4b0ee6452e5e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 02:41:22.140359 30379 raft_consensus.cc:482] T 00000000000000000000000000000000 P a2e16a9e23b24d8fa7fb4b0ee6452e5e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 02:41:22.140480 30379 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a2e16a9e23b24d8fa7fb4b0ee6452e5e [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:41:22.143100 30379 raft_consensus.cc:504] T 00000000000000000000000000000000 P a2e16a9e23b24d8fa7fb4b0ee6452e5e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2e16a9e23b24d8fa7fb4b0ee6452e5e" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 37605 } }
I20260812 02:41:22.143568 30379 leader_election.cc:302] T 00000000000000000000000000000000 P a2e16a9e23b24d8fa7fb4b0ee6452e5e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a2e16a9e23b24d8fa7fb4b0ee6452e5e; no voters:
I20260812 02:41:22.144093 30379 leader_election.cc:288] T 00000000000000000000000000000000 P a2e16a9e23b24d8fa7fb4b0ee6452e5e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 02:41:22.144263 30385 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a2e16a9e23b24d8fa7fb4b0ee6452e5e [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:41:22.145051 30385 raft_consensus.cc:686] T 00000000000000000000000000000000 P a2e16a9e23b24d8fa7fb4b0ee6452e5e [term 1 LEADER]: Becoming Leader. State: Replica: a2e16a9e23b24d8fa7fb4b0ee6452e5e, State: Running, Role: LEADER
I20260812 02:41:22.145630 30385 consensus_queue.cc:237] T 00000000000000000000000000000000 P a2e16a9e23b24d8fa7fb4b0ee6452e5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2e16a9e23b24d8fa7fb4b0ee6452e5e" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 37605 } }
I20260812 02:41:22.146387 30379 sys_catalog.cc:564] T 00000000000000000000000000000000 P a2e16a9e23b24d8fa7fb4b0ee6452e5e [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:41:22.148859 30387 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2e16a9e23b24d8fa7fb4b0ee6452e5e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a2e16a9e23b24d8fa7fb4b0ee6452e5e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2e16a9e23b24d8fa7fb4b0ee6452e5e" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 37605 } } }
I20260812 02:41:22.148885 30388 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2e16a9e23b24d8fa7fb4b0ee6452e5e [sys.catalog]: SysCatalogTable state changed. Reason: New leader a2e16a9e23b24d8fa7fb4b0ee6452e5e. Latest consensus state: current_term: 1 leader_uuid: "a2e16a9e23b24d8fa7fb4b0ee6452e5e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2e16a9e23b24d8fa7fb4b0ee6452e5e" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 37605 } } }
I20260812 02:41:22.149353 30387 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2e16a9e23b24d8fa7fb4b0ee6452e5e [sys.catalog]: This master's current role is: LEADER
I20260812 02:41:22.152865 30388 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2e16a9e23b24d8fa7fb4b0ee6452e5e [sys.catalog]: This master's current role is: LEADER
I20260812 02:41:22.154245 30395 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:41:22.160852 30395 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 02:41:22.171657 30395 catalog_manager.cc:1347] Generated new cluster ID: 3cb6f482ee6a4a55a1c1002e22390ae2
I20260812 02:41:22.171773 30395 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 02:41:22.187288 30395 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 02:41:22.188561 30395 catalog_manager.cc:1504] Loading token signing keys...
I20260812 02:41:22.195914 30395 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a2e16a9e23b24d8fa7fb4b0ee6452e5e: Generated new TSK 0
I20260812 02:41:22.196734 30395 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 02:41:22.201967 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502156971023-7698-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502156971023-7698-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502156971023-7698-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502156971023-7698-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.129:0
--local_ip_for_outbound_sockets=127.7.132.129
--webserver_interface=127.7.132.129
--webserver_port=0
--tserver_master_addrs=127.7.132.190:37605
--builtin_ntp_servers=127.7.132.148:39375
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 02:41:22.557541 30408 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:41:22.557803 30408 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:41:22.557976 30408 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:41:22.565299 30408 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:41:22.565389 30408 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:41:22.565580 30408 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.129
I20260812 02:41:22.575103 30408 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:39375
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502156971023-7698-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502156971023-7698-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502156971023-7698-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.132.129
--webserver_port=0
--tserver_master_addrs=127.7.132.190:37605
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.129
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502156971023-7698-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:41:22.576476 30408 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:41:22.578460 30408 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:41:22.591938 30415 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:41:22.592033 30416 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:41:22.592054 30419 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:41:22.593143 30408 server_base.cc:1034] running on GCE node
I20260812 02:41:22.593734 30408 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:41:22.594738 30408 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:41:22.595950 30408 hybrid_clock.cc:648] HybridClock initialized: now 1786502482595899 us; error 40 us; skew 500 ppm
I20260812 02:41:22.596295 30408 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 02:41:22.598584 30408 webserver.cc:460] Webserver started at http://127.7.132.129:42117/ using document root <none> and password file <none>
I20260812 02:41:22.599273 30408 fs_manager.cc:359] Metadata directory not provided
I20260812 02:41:22.599391 30408 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:41:22.599735 30408 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:41:22.602739 30408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502156971023-7698-0/minicluster-data/ts-0/data/instance:
uuid: "9058cc1382ac496780afcc0e6f51d87a"
format_stamp: "Formatted at 2026-08-12 02:41:22 on dist-test-slave-mn4r"
server_key: "496a79d3838f1398b17952383b653211"
server_key_iv: "1828d679fd59c0440f81088f5ed2573e"
server_key_version: "encryptionkey@0"
I20260812 02:41:22.603528 30408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502156971023-7698-0/minicluster-data/ts-0/wal/instance:
uuid: "9058cc1382ac496780afcc0e6f51d87a"
format_stamp: "Formatted at 2026-08-12 02:41:22 on dist-test-slave-mn4r"
server_key: "496a79d3838f1398b17952383b653211"
server_key_iv: "1828d679fd59c0440f81088f5ed2573e"
server_key_version: "encryptionkey@0"
I20260812 02:41:22.608439 30408 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 02:41:22.611847 30427 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:41:22.613149 30408 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 02:41:22.613320 30408 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502156971023-7698-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502156971023-7698-0/minicluster-data/ts-0/wal
uuid: "9058cc1382ac496780afcc0e6f51d87a"
format_stamp: "Formatted at 2026-08-12 02:41:22 on dist-test-slave-mn4r"
server_key: "496a79d3838f1398b17952383b653211"
server_key_iv: "1828d679fd59c0440f81088f5ed2573e"
server_key_version: "encryptionkey@0"
I20260812 02:41:22.613538 30408 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502156971023-7698-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502156971023-7698-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502156971023-7698-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:41:22.641198 30408 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:41:22.642189 30408 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:41:22.642516 30408 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:41:22.643501 30408 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:41:22.644960 30408 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:41:22.645056 30408 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:41:22.645141 30408 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:41:22.645195 30408 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:41:22.679317 30408 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.129:40655
I20260812 02:41:22.679397 30560 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.129:40655 every 8 connection(s)
I20260812 02:41:22.680711 30408 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502156971023-7698-0/minicluster-data/ts-0/data/info.pb
I20260812 02:41:22.683378 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 30408
I20260812 02:41:22.683570 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502156971023-7698-0/minicluster-data/ts-0/wal/instance
I20260812 02:41:22.684116 7698 external_mini_cluster.cc:1428] Setting key 634053f9a9a539b29b537812114f183b
I20260812 02:41:22.687311 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502156971023-7698-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502156971023-7698-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502156971023-7698-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502156971023-7698-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.130:0
--local_ip_for_outbound_sockets=127.7.132.130
--webserver_interface=127.7.132.130
--webserver_port=0
--tserver_master_addrs=127.7.132.190:37605
--builtin_ntp_servers=127.7.132.148:39375
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 02:41:22.695089 30563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:22.683565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:58795 (local address 127.7.132.190:37605)
0812 02:41:22.684113 (+ 548us) server_negotiation.cc:206] Beginning negotiation
0812 02:41:22.684117 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 02:41:22.685645 (+ 1528us) server_negotiation.cc:407] Connection header received
0812 02:41:22.687866 (+ 2221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:41:22.687874 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:41:22.687967 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:41:22.688091 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:41:22.689146 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:22.690328 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:22.691837 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:22.692542 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:22.692702 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:22.693995 (+ 1293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:41:22.694004 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:41:22.694011 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:41:22.694038 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:41:22.694130 (+ 92us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:41:22.694151 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:41:22.694242 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0812 02:41:22.694472 (+ 230us) server_negotiation.cc:299] Negotiation successful
0812 02:41:22.694591 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":433,"thread_start_us":282,"threads_started":1}
I20260812 02:41:22.695981 30564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:22.684285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:37605 (local address 127.7.132.129:58795)
0812 02:41:22.685459 (+ 1174us) negotiation.cc:107] Waiting for socket to connect
0812 02:41:22.685518 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0812 02:41:22.687586 (+ 2068us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:41:22.688290 (+ 704us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:41:22.688298 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:41:22.688386 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:41:22.688956 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:41:22.688972 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:41:22.690466 (+ 1494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:41:22.690470 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:41:22.691684 (+ 1214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:41:22.691696 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:41:22.692753 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:41:22.692757 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:41:22.692954 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:22.693678 (+ 724us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 02:41:22.693681 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:41:22.693726 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 02:41:22.693801 (+ 75us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:41:22.694262 (+ 461us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:41:22.694271 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:41:22.694277 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 02:41:22.694430 (+ 153us) client_negotiation.cc:240] Negotiation successful
0812 02:41:22.694734 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":602,"thread_start_us":365,"threads_started":1}
I20260812 02:41:22.697675 30561 heartbeater.cc:347] Connected to a master server at 127.7.132.190:37605
I20260812 02:41:22.698053 30561 heartbeater.cc:464] Registering TS with master...
I20260812 02:41:22.698868 30561 heartbeater.cc:511] Master 127.7.132.190:37605 requested a full tablet report, sending...
I20260812 02:41:22.700963 30338 ts_manager.cc:194] Registered new tserver with Master: 9058cc1382ac496780afcc0e6f51d87a (127.7.132.129:40655)
I20260812 02:41:22.703289 30338 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.132.129:58795
W20260812 02:41:23.040515 30565 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:41:23.040815 30565 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:41:23.040912 30565 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:41:23.049150 30565 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:41:23.049300 30565 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:41:23.049454 30565 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.130
I20260812 02:41:23.059779 30565 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:39375
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502156971023-7698-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502156971023-7698-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502156971023-7698-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.132.130
--webserver_port=0
--tserver_master_addrs=127.7.132.190:37605
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.130
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502156971023-7698-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:41:23.061384 30565 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:41:23.063356 30565 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:41:23.077005 30573 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:41:23.077101 30570 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:41:23.077083 30571 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:41:23.077903 30565 server_base.cc:1034] running on GCE node
I20260812 02:41:23.078557 30565 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:41:23.079476 30565 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:41:23.080714 30565 hybrid_clock.cc:648] HybridClock initialized: now 1786502483080652 us; error 29 us; skew 500 ppm
I20260812 02:41:23.081106 30565 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 02:41:23.083571 30565 webserver.cc:460] Webserver started at http://127.7.132.130:46297/ using document root <none> and password file <none>
I20260812 02:41:23.084285 30565 fs_manager.cc:359] Metadata directory not provided
I20260812 02:41:23.084409 30565 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:41:23.084748 30565 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:41:23.087836 30565 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502156971023-7698-0/minicluster-data/ts-1/data/instance:
uuid: "6a24b3e160054e808fb901f47801c4cb"
format_stamp: "Formatted at 2026-08-12 02:41:23 on dist-test-slave-mn4r"
server_key: "37a98a1ff53a3ff54f3cfb2e89818d4a"
server_key_iv: "71287a7caa33429b9e42d770d1462cc2"
server_key_version: "encryptionkey@0"
I20260812 02:41:23.088655 30565 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502156971023-7698-0/minicluster-data/ts-1/wal/instance:
uuid: "6a24b3e160054e808fb901f47801c4cb"
format_stamp: "Formatted at 2026-08-12 02:41:23 on dist-test-slave-mn4r"
server_key: "37a98a1ff53a3ff54f3cfb2e89818d4a"
server_key_iv: "71287a7caa33429b9e42d770d1462cc2"
server_key_version: "encryptionkey@0"
I20260812 02:41:23.093921 30565 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260812 02:41:23.097628 30581 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:41:23.098979 30565 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 02:41:23.099162 30565 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502156971023-7698-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502156971023-7698-0/minicluster-data/ts-1/wal
uuid: "6a24b3e160054e808fb901f47801c4cb"
format_stamp: "Formatted at 2026-08-12 02:41:23 on dist-test-slave-mn4r"
server_key: "37a98a1ff53a3ff54f3cfb2e89818d4a"
server_key_iv: "71287a7caa33429b9e42d770d1462cc2"
server_key_version: "encryptionkey@0"
I20260812 02:41:23.099397 30565 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502156971023-7698-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502156971023-7698-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502156971023-7698-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:41:23.117720 30565 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:41:23.118788 30565 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:41:23.119112 30565 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:41:23.120294 30565 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:41:23.122198 30565 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:41:23.122315 30565 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:41:23.122417 30565 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:41:23.122498 30565 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:41:23.156980 30565 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.130:37793
I20260812 02:41:23.157024 30717 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.130:37793 every 8 connection(s)
I20260812 02:41:23.158429 30565 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502156971023-7698-0/minicluster-data/ts-1/data/info.pb
I20260812 02:41:23.162566 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 30565
I20260812 02:41:23.162768 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502156971023-7698-0/minicluster-data/ts-1/wal/instance
I20260812 02:41:23.163424 7698 external_mini_cluster.cc:1428] Setting key 1d83a035df1015df6516d104a3aba760
I20260812 02:41:23.166555 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502156971023-7698-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502156971023-7698-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502156971023-7698-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502156971023-7698-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.131:0
--local_ip_for_outbound_sockets=127.7.132.131
--webserver_interface=127.7.132.131
--webserver_port=0
--tserver_master_addrs=127.7.132.190:37605
--builtin_ntp_servers=127.7.132.148:39375
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 02:41:23.172111 30563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:23.161204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:35291 (local address 127.7.132.190:37605)
0812 02:41:23.161392 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0812 02:41:23.161396 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 02:41:23.162967 (+ 1571us) server_negotiation.cc:407] Connection header received
0812 02:41:23.164955 (+ 1988us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:41:23.164959 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:41:23.165023 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:41:23.165109 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:41:23.166277 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:23.167402 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:23.168935 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:23.169643 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:23.169786 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:23.171174 (+ 1388us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:41:23.171183 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:41:23.171190 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:41:23.171216 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:41:23.171258 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:41:23.171276 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:41:23.171373 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0812 02:41:23.171684 (+ 311us) server_negotiation.cc:299] Negotiation successful
0812 02:41:23.171783 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 02:41:23.173096 30721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:23.161735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:37605 (local address 127.7.132.130:35291)
0812 02:41:23.162780 (+ 1045us) negotiation.cc:107] Waiting for socket to connect
0812 02:41:23.162838 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0812 02:41:23.164675 (+ 1837us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:41:23.165387 (+ 712us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:41:23.165396 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:41:23.165479 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:41:23.166051 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:41:23.166068 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:41:23.167553 (+ 1485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:41:23.167556 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:41:23.168784 (+ 1228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:41:23.168797 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:41:23.169849 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:41:23.169854 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:41:23.170048 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:23.170857 (+ 809us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 02:41:23.170859 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:41:23.170906 (+ 47us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 02:41:23.170982 (+ 76us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:41:23.171376 (+ 394us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:41:23.171384 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:41:23.171389 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 02:41:23.171657 (+ 268us) client_negotiation.cc:240] Negotiation successful
0812 02:41:23.172046 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":517,"thread_start_us":353,"threads_started":1}
I20260812 02:41:23.174597 30718 heartbeater.cc:347] Connected to a master server at 127.7.132.190:37605
I20260812 02:41:23.174921 30718 heartbeater.cc:464] Registering TS with master...
I20260812 02:41:23.175681 30718 heartbeater.cc:511] Master 127.7.132.190:37605 requested a full tablet report, sending...
I20260812 02:41:23.177194 30338 ts_manager.cc:194] Registered new tserver with Master: 6a24b3e160054e808fb901f47801c4cb (127.7.132.130:37793)
I20260812 02:41:23.178459 30338 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.132.130:35291
W20260812 02:41:23.545590 30722 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:41:23.546095 30722 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:41:23.546262 30722 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:41:23.554818 30722 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:41:23.554963 30722 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:41:23.555076 30722 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.131
I20260812 02:41:23.564226 30722 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:39375
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502156971023-7698-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502156971023-7698-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502156971023-7698-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.132.131
--webserver_port=0
--tserver_master_addrs=127.7.132.190:37605
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.131
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502156971023-7698-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:41:23.565492 30722 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:41:23.567305 30722 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:41:23.580767 30729 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:41:23.580869 30732 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:41:23.580801 30730 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:41:23.581526 30722 server_base.cc:1034] running on GCE node
I20260812 02:41:23.582259 30722 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:41:23.583214 30722 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:41:23.584450 30722 hybrid_clock.cc:648] HybridClock initialized: now 1786502483584399 us; error 61 us; skew 500 ppm
I20260812 02:41:23.584770 30722 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 02:41:23.587095 30722 webserver.cc:460] Webserver started at http://127.7.132.131:41889/ using document root <none> and password file <none>
I20260812 02:41:23.587812 30722 fs_manager.cc:359] Metadata directory not provided
I20260812 02:41:23.587949 30722 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:41:23.588276 30722 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:41:23.591241 30722 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502156971023-7698-0/minicluster-data/ts-2/data/instance:
uuid: "1462f7a54ea54853a62604e14c997f41"
format_stamp: "Formatted at 2026-08-12 02:41:23 on dist-test-slave-mn4r"
server_key: "2d6b492621976439b7440d1c55ff0537"
server_key_iv: "04958d32b88ed922940b2bdb68d5fe73"
server_key_version: "encryptionkey@0"
I20260812 02:41:23.592027 30722 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502156971023-7698-0/minicluster-data/ts-2/wal/instance:
uuid: "1462f7a54ea54853a62604e14c997f41"
format_stamp: "Formatted at 2026-08-12 02:41:23 on dist-test-slave-mn4r"
server_key: "2d6b492621976439b7440d1c55ff0537"
server_key_iv: "04958d32b88ed922940b2bdb68d5fe73"
server_key_version: "encryptionkey@0"
I20260812 02:41:23.597275 30722 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260812 02:41:23.600976 30739 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:41:23.602402 30722 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 02:41:23.602582 30722 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502156971023-7698-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502156971023-7698-0/minicluster-data/ts-2/wal
uuid: "1462f7a54ea54853a62604e14c997f41"
format_stamp: "Formatted at 2026-08-12 02:41:23 on dist-test-slave-mn4r"
server_key: "2d6b492621976439b7440d1c55ff0537"
server_key_iv: "04958d32b88ed922940b2bdb68d5fe73"
server_key_version: "encryptionkey@0"
I20260812 02:41:23.602846 30722 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502156971023-7698-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502156971023-7698-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502156971023-7698-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:41:23.623250 30722 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:41:23.624153 30722 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:41:23.624491 30722 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:41:23.625517 30722 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:41:23.627341 30722 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:41:23.627466 30722 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:41:23.627601 30722 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:41:23.627699 30722 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:41:23.662786 30722 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.131:36957
I20260812 02:41:23.662827 30868 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.131:36957 every 8 connection(s)
I20260812 02:41:23.664122 30722 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502156971023-7698-0/minicluster-data/ts-2/data/info.pb
I20260812 02:41:23.670509 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 30722
I20260812 02:41:23.670702 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502156971023-7698-0/minicluster-data/ts-2/wal/instance
I20260812 02:41:23.671260 7698 external_mini_cluster.cc:1428] Setting key 0741630c0bbd4e139d6e27367fd52f1d
I20260812 02:41:23.677369 30563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:23.666959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:44707 (local address 127.7.132.190:37605)
0812 02:41:23.667163 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0812 02:41:23.667167 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 02:41:23.668639 (+ 1472us) server_negotiation.cc:407] Connection header received
0812 02:41:23.670484 (+ 1845us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:41:23.670493 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:41:23.670591 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:41:23.670726 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:41:23.672029 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:23.673123 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:23.674980 (+ 1857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:23.675388 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:23.675486 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:23.676727 (+ 1241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:41:23.676732 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:41:23.676735 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:41:23.676758 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:41:23.676790 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:41:23.676802 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:41:23.676872 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 02:41:23.677062 (+ 190us) server_negotiation.cc:299] Negotiation successful
0812 02:41:23.677139 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260812 02:41:23.678563 30871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:23.667527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:37605 (local address 127.7.132.131:44707)
0812 02:41:23.668493 (+ 966us) negotiation.cc:107] Waiting for socket to connect
0812 02:41:23.668532 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 02:41:23.670201 (+ 1669us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:41:23.670979 (+ 778us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:41:23.670987 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:41:23.671095 (+ 108us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:41:23.671857 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:41:23.671885 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:41:23.673301 (+ 1416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:41:23.673305 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:41:23.674847 (+ 1542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:41:23.674859 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:41:23.675575 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:41:23.675580 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:41:23.675770 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:23.676416 (+ 646us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 02:41:23.676418 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:41:23.676457 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 02:41:23.676538 (+ 81us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:41:23.676885 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:41:23.676892 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:41:23.676898 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 02:41:23.677091 (+ 193us) client_negotiation.cc:240] Negotiation successful
0812 02:41:23.677373 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":523,"thread_start_us":284,"threads_started":1}
I20260812 02:41:23.680061 30869 heartbeater.cc:347] Connected to a master server at 127.7.132.190:37605
I20260812 02:41:23.680377 30869 heartbeater.cc:464] Registering TS with master...
I20260812 02:41:23.681110 30869 heartbeater.cc:511] Master 127.7.132.190:37605 requested a full tablet report, sending...
I20260812 02:41:23.682696 30338 ts_manager.cc:194] Registered new tserver with Master: 1462f7a54ea54853a62604e14c997f41 (127.7.132.131:36957)
I20260812 02:41:23.683760 30338 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.132.131:44707
I20260812 02:41:23.686725 7698 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 02:41:23.696940 30563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:23.690740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40600 (local address 127.7.132.190:37605)
0812 02:41:23.690951 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0812 02:41:23.690958 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:41:23.691496 (+ 538us) server_negotiation.cc:407] Connection header received
0812 02:41:23.691666 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:41:23.691671 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:41:23.691747 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:41:23.691836 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:41:23.692363 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:23.693083 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:23.694489 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:23.694903 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:23.695000 (+ 97us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:23.696250 (+ 1250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:41:23.696254 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:41:23.696257 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:41:23.696274 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:41:23.696304 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:41:23.696321 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:41:23.696432 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0812 02:41:23.696671 (+ 239us) server_negotiation.cc:299] Negotiation successful
0812 02:41:23.696715 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260812 02:41:23.700984 30338 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40600:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260812 02:41:23.707139 30561 heartbeater.cc:503] Master 127.7.132.190:37605 was elected leader, sending a full tablet report...
W20260812 02:41:23.708262 30338 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 02:41:23.733067 30885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:23.720750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58620 (local address 127.7.132.129:40655)
0812 02:41:23.722518 (+ 1768us) server_negotiation.cc:206] Beginning negotiation
0812 02:41:23.722529 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 02:41:23.722562 (+ 33us) server_negotiation.cc:407] Connection header received
0812 02:41:23.722701 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:41:23.722709 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:41:23.722986 (+ 277us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:41:23.724509 (+ 1523us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:41:23.724835 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:23.726986 (+ 2151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:23.730365 (+ 3379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:23.731776 (+ 1411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:23.731921 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:23.732234 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0812 02:41:23.732460 (+ 226us) server_negotiation.cc:299] Negotiation successful
0812 02:41:23.732668 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1589,"thread_start_us":4994,"threads_started":1}
I20260812 02:41:23.733094 30886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:23.720496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:40655 (local address 127.0.0.1:58620)
0812 02:41:23.721772 (+ 1276us) negotiation.cc:107] Waiting for socket to connect
0812 02:41:23.721821 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 02:41:23.722033 (+ 212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:41:23.723267 (+ 1234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:41:23.723276 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:41:23.723311 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:41:23.723924 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:41:23.723942 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:41:23.727193 (+ 3251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:41:23.727205 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:41:23.730144 (+ 2939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:41:23.730166 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:41:23.731973 (+ 1807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:41:23.731979 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:41:23.732198 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:23.732258 (+ 60us) client_negotiation.cc:769] Sending connection context
0812 02:41:23.732404 (+ 146us) client_negotiation.cc:240] Negotiation successful
0812 02:41:23.732650 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":942,"mutex_wait_us":4,"thread_start_us":8303,"threads_started":1}
I20260812 02:41:23.737051 30888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:23.720815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.131:36957 (local address 127.0.0.1:59424)
0812 02:41:23.728738 (+ 7923us) negotiation.cc:107] Waiting for socket to connect
0812 02:41:23.728761 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0812 02:41:23.728916 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:41:23.729631 (+ 715us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:41:23.729640 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:41:23.729665 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:41:23.730178 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:41:23.730197 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:41:23.732214 (+ 2017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:41:23.732219 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:41:23.734402 (+ 2183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:41:23.734419 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:41:23.736285 (+ 1866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:41:23.736292 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:41:23.736453 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:23.736485 (+ 32us) client_negotiation.cc:769] Sending connection context
0812 02:41:23.736582 (+ 97us) client_negotiation.cc:240] Negotiation successful
0812 02:41:23.736700 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7775,"mutex_wait_us":115,"thread_start_us":497,"threads_started":1}
I20260812 02:41:23.737181 30887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:23.721382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59424 (local address 127.7.132.131:36957)
0812 02:41:23.723893 (+ 2511us) server_negotiation.cc:206] Beginning negotiation
0812 02:41:23.723906 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 02:41:23.729070 (+ 5164us) server_negotiation.cc:407] Connection header received
0812 02:41:23.729187 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:41:23.729195 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:41:23.729445 (+ 250us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:41:23.729649 (+ 204us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:41:23.730415 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:23.732039 (+ 1624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:23.734654 (+ 2615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:23.736071 (+ 1417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:23.736212 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:23.736474 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0812 02:41:23.736641 (+ 167us) server_negotiation.cc:299] Negotiation successful
0812 02:41:23.736809 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2285,"thread_start_us":1012,"threads_started":1}
I20260812 02:41:23.738036 30884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:23.722675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.130:37793 (local address 127.0.0.1:47950)
0812 02:41:23.729084 (+ 6409us) negotiation.cc:107] Waiting for socket to connect
0812 02:41:23.729101 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 02:41:23.729226 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:41:23.729966 (+ 740us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:41:23.729972 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:41:23.730008 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:41:23.730523 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:41:23.730539 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:41:23.732345 (+ 1806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:41:23.732353 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:41:23.734945 (+ 2592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:41:23.734961 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:41:23.737073 (+ 2112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:41:23.737080 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:41:23.737463 (+ 383us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:23.737492 (+ 29us) client_negotiation.cc:769] Sending connection context
0812 02:41:23.737586 (+ 94us) client_negotiation.cc:240] Negotiation successful
0812 02:41:23.737701 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6287,"mutex_wait_us":1,"thread_start_us":5755,"threads_started":1}
I20260812 02:41:23.738225 30890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:23.723210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47950 (local address 127.7.132.130:37793)
0812 02:41:23.728221 (+ 5011us) server_negotiation.cc:206] Beginning negotiation
0812 02:41:23.728232 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 02:41:23.729222 (+ 990us) server_negotiation.cc:407] Connection header received
0812 02:41:23.729356 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:41:23.729363 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:41:23.729596 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:41:23.729790 (+ 194us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:41:23.730745 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:23.732223 (+ 1478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:23.735192 (+ 2969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:23.736592 (+ 1400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:23.736711 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:23.736919 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0812 02:41:23.737685 (+ 766us) server_negotiation.cc:299] Negotiation successful
0812 02:41:23.737856 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4815,"thread_start_us":3163,"threads_started":1}
I20260812 02:41:23.737727 30485 tablet_service.cc:1467] Processing CreateTablet for tablet 5f630de783264afe889a2adf30269a04 (DEFAULT_TABLE table=test-table [id=935cb65c954d43368f2bf1362b26facb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:41:23.739456 30485 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5f630de783264afe889a2adf30269a04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:41:23.740866 30794 tablet_service.cc:1467] Processing CreateTablet for tablet 5f630de783264afe889a2adf30269a04 (DEFAULT_TABLE table=test-table [id=935cb65c954d43368f2bf1362b26facb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:41:23.741752 30642 tablet_service.cc:1467] Processing CreateTablet for tablet 5f630de783264afe889a2adf30269a04 (DEFAULT_TABLE table=test-table [id=935cb65c954d43368f2bf1362b26facb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:41:23.742328 30794 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5f630de783264afe889a2adf30269a04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:41:23.743139 30642 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5f630de783264afe889a2adf30269a04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:41:23.751374 30891 tablet_bootstrap.cc:492] T 5f630de783264afe889a2adf30269a04 P 9058cc1382ac496780afcc0e6f51d87a: Bootstrap starting.
I20260812 02:41:23.754990 30891 tablet_bootstrap.cc:654] T 5f630de783264afe889a2adf30269a04 P 9058cc1382ac496780afcc0e6f51d87a: Neither blocks nor log segments found. Creating new log.
I20260812 02:41:23.756094 30891 log.cc:824] T 5f630de783264afe889a2adf30269a04 P 9058cc1382ac496780afcc0e6f51d87a: Log is configured to *not* fsync() on all Append() calls
I20260812 02:41:23.758970 30891 tablet_bootstrap.cc:492] T 5f630de783264afe889a2adf30269a04 P 9058cc1382ac496780afcc0e6f51d87a: No bootstrap required, opened a new log
I20260812 02:41:23.759258 30891 ts_tablet_manager.cc:1397] T 5f630de783264afe889a2adf30269a04 P 9058cc1382ac496780afcc0e6f51d87a: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260812 02:41:23.762116 30894 tablet_bootstrap.cc:492] T 5f630de783264afe889a2adf30269a04 P 6a24b3e160054e808fb901f47801c4cb: Bootstrap starting.
I20260812 02:41:23.764175 30893 tablet_bootstrap.cc:492] T 5f630de783264afe889a2adf30269a04 P 1462f7a54ea54853a62604e14c997f41: Bootstrap starting.
I20260812 02:41:23.765833 30894 tablet_bootstrap.cc:654] T 5f630de783264afe889a2adf30269a04 P 6a24b3e160054e808fb901f47801c4cb: Neither blocks nor log segments found. Creating new log.
I20260812 02:41:23.767170 30893 tablet_bootstrap.cc:654] T 5f630de783264afe889a2adf30269a04 P 1462f7a54ea54853a62604e14c997f41: Neither blocks nor log segments found. Creating new log.
I20260812 02:41:23.767128 30894 log.cc:824] T 5f630de783264afe889a2adf30269a04 P 6a24b3e160054e808fb901f47801c4cb: Log is configured to *not* fsync() on all Append() calls
I20260812 02:41:23.768339 30893 log.cc:824] T 5f630de783264afe889a2adf30269a04 P 1462f7a54ea54853a62604e14c997f41: Log is configured to *not* fsync() on all Append() calls
I20260812 02:41:23.768709 30891 raft_consensus.cc:348] T 5f630de783264afe889a2adf30269a04 P 9058cc1382ac496780afcc0e6f51d87a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9058cc1382ac496780afcc0e6f51d87a" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 40655 } } peers { permanent_uuid: "1462f7a54ea54853a62604e14c997f41" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 36957 } } peers { permanent_uuid: "6a24b3e160054e808fb901f47801c4cb" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37793 } }
I20260812 02:41:23.769178 30891 raft_consensus.cc:374] T 5f630de783264afe889a2adf30269a04 P 9058cc1382ac496780afcc0e6f51d87a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:41:23.769284 30891 raft_consensus.cc:729] T 5f630de783264afe889a2adf30269a04 P 9058cc1382ac496780afcc0e6f51d87a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9058cc1382ac496780afcc0e6f51d87a, State: Initialized, Role: FOLLOWER
I20260812 02:41:23.769953 30891 consensus_queue.cc:260] T 5f630de783264afe889a2adf30269a04 P 9058cc1382ac496780afcc0e6f51d87a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9058cc1382ac496780afcc0e6f51d87a" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 40655 } } peers { permanent_uuid: "1462f7a54ea54853a62604e14c997f41" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 36957 } } peers { permanent_uuid: "6a24b3e160054e808fb901f47801c4cb" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37793 } }
I20260812 02:41:23.770709 30894 tablet_bootstrap.cc:492] T 5f630de783264afe889a2adf30269a04 P 6a24b3e160054e808fb901f47801c4cb: No bootstrap required, opened a new log
I20260812 02:41:23.771044 30894 ts_tablet_manager.cc:1397] T 5f630de783264afe889a2adf30269a04 P 6a24b3e160054e808fb901f47801c4cb: Time spent bootstrapping tablet: real 0.014s user 0.005s sys 0.003s
I20260812 02:41:23.771551 30893 tablet_bootstrap.cc:492] T 5f630de783264afe889a2adf30269a04 P 1462f7a54ea54853a62604e14c997f41: No bootstrap required, opened a new log
I20260812 02:41:23.771844 30893 ts_tablet_manager.cc:1397] T 5f630de783264afe889a2adf30269a04 P 1462f7a54ea54853a62604e14c997f41: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260812 02:41:23.772022 30891 ts_tablet_manager.cc:1428] T 5f630de783264afe889a2adf30269a04 P 9058cc1382ac496780afcc0e6f51d87a: Time spent starting tablet: real 0.013s user 0.005s sys 0.008s
I20260812 02:41:23.780570 30894 raft_consensus.cc:348] T 5f630de783264afe889a2adf30269a04 P 6a24b3e160054e808fb901f47801c4cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9058cc1382ac496780afcc0e6f51d87a" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 40655 } } peers { permanent_uuid: "1462f7a54ea54853a62604e14c997f41" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 36957 } } peers { permanent_uuid: "6a24b3e160054e808fb901f47801c4cb" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37793 } }
I20260812 02:41:23.780568 30893 raft_consensus.cc:348] T 5f630de783264afe889a2adf30269a04 P 1462f7a54ea54853a62604e14c997f41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9058cc1382ac496780afcc0e6f51d87a" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 40655 } } peers { permanent_uuid: "1462f7a54ea54853a62604e14c997f41" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 36957 } } peers { permanent_uuid: "6a24b3e160054e808fb901f47801c4cb" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37793 } }
I20260812 02:41:23.780958 30894 raft_consensus.cc:374] T 5f630de783264afe889a2adf30269a04 P 6a24b3e160054e808fb901f47801c4cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:41:23.780972 30893 raft_consensus.cc:374] T 5f630de783264afe889a2adf30269a04 P 1462f7a54ea54853a62604e14c997f41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:41:23.781076 30893 raft_consensus.cc:729] T 5f630de783264afe889a2adf30269a04 P 1462f7a54ea54853a62604e14c997f41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1462f7a54ea54853a62604e14c997f41, State: Initialized, Role: FOLLOWER
I20260812 02:41:23.781077 30894 raft_consensus.cc:729] T 5f630de783264afe889a2adf30269a04 P 6a24b3e160054e808fb901f47801c4cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a24b3e160054e808fb901f47801c4cb, State: Initialized, Role: FOLLOWER
I20260812 02:41:23.781710 30893 consensus_queue.cc:260] T 5f630de783264afe889a2adf30269a04 P 1462f7a54ea54853a62604e14c997f41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9058cc1382ac496780afcc0e6f51d87a" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 40655 } } peers { permanent_uuid: "1462f7a54ea54853a62604e14c997f41" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 36957 } } peers { permanent_uuid: "6a24b3e160054e808fb901f47801c4cb" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37793 } }
I20260812 02:41:23.781706 30894 consensus_queue.cc:260] T 5f630de783264afe889a2adf30269a04 P 6a24b3e160054e808fb901f47801c4cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9058cc1382ac496780afcc0e6f51d87a" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 40655 } } peers { permanent_uuid: "1462f7a54ea54853a62604e14c997f41" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 36957 } } peers { permanent_uuid: "6a24b3e160054e808fb901f47801c4cb" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37793 } }
I20260812 02:41:23.782892 30718 heartbeater.cc:503] Master 127.7.132.190:37605 was elected leader, sending a full tablet report...
I20260812 02:41:23.782893 30869 heartbeater.cc:503] Master 127.7.132.190:37605 was elected leader, sending a full tablet report...
I20260812 02:41:23.783653 30893 ts_tablet_manager.cc:1428] T 5f630de783264afe889a2adf30269a04 P 1462f7a54ea54853a62604e14c997f41: Time spent starting tablet: real 0.012s user 0.007s sys 0.004s
I20260812 02:41:23.783663 30894 ts_tablet_manager.cc:1428] T 5f630de783264afe889a2adf30269a04 P 6a24b3e160054e808fb901f47801c4cb: Time spent starting tablet: real 0.012s user 0.004s sys 0.006s
W20260812 02:41:23.910857 30720 tablet.cc:2369] T 5f630de783264afe889a2adf30269a04 P 6a24b3e160054e808fb901f47801c4cb: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 02:41:23.915971 30870 tablet.cc:2369] T 5f630de783264afe889a2adf30269a04 P 1462f7a54ea54853a62604e14c997f41: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 02:41:23.933365 30562 tablet.cc:2369] T 5f630de783264afe889a2adf30269a04 P 9058cc1382ac496780afcc0e6f51d87a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 02:41:24.087759 30898 raft_consensus.cc:482] T 5f630de783264afe889a2adf30269a04 P 1462f7a54ea54853a62604e14c997f41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 02:41:24.088122 30898 raft_consensus.cc:504] T 5f630de783264afe889a2adf30269a04 P 1462f7a54ea54853a62604e14c997f41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9058cc1382ac496780afcc0e6f51d87a" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 40655 } } peers { permanent_uuid: "1462f7a54ea54853a62604e14c997f41" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 36957 } } peers { permanent_uuid: "6a24b3e160054e808fb901f47801c4cb" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37793 } }
I20260812 02:41:24.090301 30898 leader_election.cc:288] T 5f630de783264afe889a2adf30269a04 P 1462f7a54ea54853a62604e14c997f41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9058cc1382ac496780afcc0e6f51d87a (127.7.132.129:40655), 6a24b3e160054e808fb901f47801c4cb (127.7.132.130:37793)
I20260812 02:41:24.096764 30885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:24.090842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:34405 (local address 127.7.132.129:40655)
0812 02:41:24.091070 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0812 02:41:24.091076 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:41:24.091096 (+ 20us) server_negotiation.cc:407] Connection header received
0812 02:41:24.091176 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:41:24.091182 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:41:24.091279 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:41:24.091377 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:41:24.092201 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:24.092970 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:24.095047 (+ 2077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:24.095839 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:24.096023 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:24.096103 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 02:41:24.096368 (+ 265us) server_negotiation.cc:299] Negotiation successful
0812 02:41:24.096498 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260812 02:41:24.096794 30871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:24.090516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:40655 (local address 127.7.132.131:34405)
0812 02:41:24.090839 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0812 02:41:24.090860 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 02:41:24.090985 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:41:24.091824 (+ 839us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:41:24.091829 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:41:24.091861 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:41:24.092023 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:41:24.092030 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:41:24.093115 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:41:24.093122 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:41:24.094866 (+ 1744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:41:24.094878 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:41:24.095999 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:41:24.096006 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:41:24.096206 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:24.096230 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 02:41:24.096338 (+ 108us) client_negotiation.cc:240] Negotiation successful
0812 02:41:24.096494 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":199}
I20260812 02:41:24.097209 30890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:24.090806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:43961 (local address 127.7.132.130:37793)
0812 02:41:24.090989 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0812 02:41:24.090996 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:41:24.092071 (+ 1075us) server_negotiation.cc:407] Connection header received
0812 02:41:24.092281 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:41:24.092286 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:41:24.092357 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:41:24.092457 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:41:24.092970 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:24.093722 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:24.095548 (+ 1826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:24.096313 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:24.096493 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:24.096575 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 02:41:24.096836 (+ 261us) server_negotiation.cc:299] Negotiation successful
0812 02:41:24.096964 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 02:41:24.097301 30901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:24.090516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.130:37793 (local address 127.7.132.131:43961)
0812 02:41:24.091947 (+ 1431us) negotiation.cc:107] Waiting for socket to connect
0812 02:41:24.091961 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 02:41:24.092133 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:41:24.092471 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:41:24.092475 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:41:24.092500 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:41:24.092818 (+ 318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:41:24.092826 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:41:24.093877 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:41:24.093881 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:41:24.095348 (+ 1467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:41:24.095361 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:41:24.096491 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:41:24.096497 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:41:24.096673 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:24.096697 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 02:41:24.096811 (+ 114us) client_negotiation.cc:240] Negotiation successful
0812 02:41:24.096934 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1312,"mutex_wait_us":493,"thread_start_us":345,"threads_started":1}
I20260812 02:41:24.098237 30511 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f630de783264afe889a2adf30269a04" candidate_uuid: "1462f7a54ea54853a62604e14c997f41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9058cc1382ac496780afcc0e6f51d87a" is_pre_election: true
I20260812 02:41:24.098261 30667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f630de783264afe889a2adf30269a04" candidate_uuid: "1462f7a54ea54853a62604e14c997f41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a24b3e160054e808fb901f47801c4cb" is_pre_election: true
I20260812 02:41:24.098774 30511 raft_consensus.cc:2454] T 5f630de783264afe889a2adf30269a04 P 9058cc1382ac496780afcc0e6f51d87a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1462f7a54ea54853a62604e14c997f41 in term 0.
I20260812 02:41:24.098776 30667 raft_consensus.cc:2454] T 5f630de783264afe889a2adf30269a04 P 6a24b3e160054e808fb901f47801c4cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1462f7a54ea54853a62604e14c997f41 in term 0.
I20260812 02:41:24.099453 30742 leader_election.cc:302] T 5f630de783264afe889a2adf30269a04 P 1462f7a54ea54853a62604e14c997f41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 1462f7a54ea54853a62604e14c997f41, 6a24b3e160054e808fb901f47801c4cb, 9058cc1382ac496780afcc0e6f51d87a; no voters:
I20260812 02:41:24.099874 30898 raft_consensus.cc:2781] T 5f630de783264afe889a2adf30269a04 P 1462f7a54ea54853a62604e14c997f41 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 02:41:24.100026 30898 raft_consensus.cc:482] T 5f630de783264afe889a2adf30269a04 P 1462f7a54ea54853a62604e14c997f41 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 02:41:24.100150 30898 raft_consensus.cc:3037] T 5f630de783264afe889a2adf30269a04 P 1462f7a54ea54853a62604e14c997f41 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:41:24.102442 30898 raft_consensus.cc:504] T 5f630de783264afe889a2adf30269a04 P 1462f7a54ea54853a62604e14c997f41 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9058cc1382ac496780afcc0e6f51d87a" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 40655 } } peers { permanent_uuid: "1462f7a54ea54853a62604e14c997f41" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 36957 } } peers { permanent_uuid: "6a24b3e160054e808fb901f47801c4cb" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37793 } }
I20260812 02:41:24.103127 30898 leader_election.cc:288] T 5f630de783264afe889a2adf30269a04 P 1462f7a54ea54853a62604e14c997f41 [CANDIDATE]: Term 1 election: Requested vote from peers 9058cc1382ac496780afcc0e6f51d87a (127.7.132.129:40655), 6a24b3e160054e808fb901f47801c4cb (127.7.132.130:37793)
I20260812 02:41:24.103627 30511 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f630de783264afe889a2adf30269a04" candidate_uuid: "1462f7a54ea54853a62604e14c997f41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9058cc1382ac496780afcc0e6f51d87a"
I20260812 02:41:24.103659 30667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f630de783264afe889a2adf30269a04" candidate_uuid: "1462f7a54ea54853a62604e14c997f41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a24b3e160054e808fb901f47801c4cb"
I20260812 02:41:24.103842 30511 raft_consensus.cc:3037] T 5f630de783264afe889a2adf30269a04 P 9058cc1382ac496780afcc0e6f51d87a [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:41:24.103849 30667 raft_consensus.cc:3037] T 5f630de783264afe889a2adf30269a04 P 6a24b3e160054e808fb901f47801c4cb [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:41:24.106549 30667 raft_consensus.cc:2454] T 5f630de783264afe889a2adf30269a04 P 6a24b3e160054e808fb901f47801c4cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1462f7a54ea54853a62604e14c997f41 in term 1.
I20260812 02:41:24.107132 30742 leader_election.cc:302] T 5f630de783264afe889a2adf30269a04 P 1462f7a54ea54853a62604e14c997f41 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1462f7a54ea54853a62604e14c997f41, 6a24b3e160054e808fb901f47801c4cb; no voters:
I20260812 02:41:24.107442 30898 raft_consensus.cc:2781] T 5f630de783264afe889a2adf30269a04 P 1462f7a54ea54853a62604e14c997f41 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:41:24.107780 30511 raft_consensus.cc:2454] T 5f630de783264afe889a2adf30269a04 P 9058cc1382ac496780afcc0e6f51d87a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1462f7a54ea54853a62604e14c997f41 in term 1.
I20260812 02:41:24.108022 30898 raft_consensus.cc:686] T 5f630de783264afe889a2adf30269a04 P 1462f7a54ea54853a62604e14c997f41 [term 1 LEADER]: Becoming Leader. State: Replica: 1462f7a54ea54853a62604e14c997f41, State: Running, Role: LEADER
I20260812 02:41:24.108570 30898 consensus_queue.cc:237] T 5f630de783264afe889a2adf30269a04 P 1462f7a54ea54853a62604e14c997f41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9058cc1382ac496780afcc0e6f51d87a" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 40655 } } peers { permanent_uuid: "1462f7a54ea54853a62604e14c997f41" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 36957 } } peers { permanent_uuid: "6a24b3e160054e808fb901f47801c4cb" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37793 } }
I20260812 02:41:24.113925 30336 catalog_manager.cc:5495] T 5f630de783264afe889a2adf30269a04 P 1462f7a54ea54853a62604e14c997f41 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1462f7a54ea54853a62604e14c997f41 (127.7.132.131). New cstate: current_term: 1 leader_uuid: "1462f7a54ea54853a62604e14c997f41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9058cc1382ac496780afcc0e6f51d87a" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 40655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1462f7a54ea54853a62604e14c997f41" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 36957 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a24b3e160054e808fb901f47801c4cb" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37793 } health_report { overall_health: UNKNOWN } } }
I20260812 02:41:24.192444 30887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:24.186466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59440 (local address 127.7.132.131:36957)
0812 02:41:24.186679 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0812 02:41:24.186684 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:41:24.186702 (+ 18us) server_negotiation.cc:407] Connection header received
0812 02:41:24.186783 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:41:24.186787 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:41:24.186895 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:41:24.187024 (+ 129us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 02:41:24.187399 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:24.188343 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:24.190200 (+ 1857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:24.190633 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:24.190734 (+ 101us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:24.191258 (+ 524us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 02:41:24.191786 (+ 528us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 02:41:24.191920 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0812 02:41:24.192106 (+ 186us) server_negotiation.cc:299] Negotiation successful
0812 02:41:24.192175 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260812 02:41:24.200860 30667 raft_consensus.cc:1261] T 5f630de783264afe889a2adf30269a04 P 6a24b3e160054e808fb901f47801c4cb [term 1 FOLLOWER]: Refusing update from remote peer 1462f7a54ea54853a62604e14c997f41: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:41:24.200889 30511 raft_consensus.cc:1261] T 5f630de783264afe889a2adf30269a04 P 9058cc1382ac496780afcc0e6f51d87a [term 1 FOLLOWER]: Refusing update from remote peer 1462f7a54ea54853a62604e14c997f41: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:41:24.201862 30902 consensus_queue.cc:1035] T 5f630de783264afe889a2adf30269a04 P 1462f7a54ea54853a62604e14c997f41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9058cc1382ac496780afcc0e6f51d87a" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 40655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:41:24.202330 30898 consensus_queue.cc:1035] T 5f630de783264afe889a2adf30269a04 P 1462f7a54ea54853a62604e14c997f41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a24b3e160054e808fb901f47801c4cb" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37793 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:41:24.216382 30909 mvcc.cc:205] Tried to move back new op lower bound from 7317514175276302336 to 7317514174912794624. Current Snapshot: MvccSnapshot[applied={T|T < 7317514175276302336}]
I20260812 02:41:24.235045 30337 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:40600:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 02:41:24.235306 30337 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:40600:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 02:41:24.239166 30337 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a2e16a9e23b24d8fa7fb4b0ee6452e5e: Sending DeleteTablet for 3 replicas of tablet 5f630de783264afe889a2adf30269a04
I20260812 02:41:24.240430 30642 tablet_service.cc:1514] Processing DeleteTablet for tablet 5f630de783264afe889a2adf30269a04 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:41:24 UTC) from {username='slave'} at 127.0.0.1:47950
I20260812 02:41:24.240449 30794 tablet_service.cc:1514] Processing DeleteTablet for tablet 5f630de783264afe889a2adf30269a04 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:41:24 UTC) from {username='slave'} at 127.0.0.1:59424
I20260812 02:41:24.240437 30485 tablet_service.cc:1514] Processing DeleteTablet for tablet 5f630de783264afe889a2adf30269a04 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:41:24 UTC) from {username='slave'} at 127.0.0.1:58620
I20260812 02:41:24.246212 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 30408
I20260812 02:41:24.260478 30919 tablet_replica.cc:330] T 5f630de783264afe889a2adf30269a04 P 1462f7a54ea54853a62604e14c997f41: stopping tablet replica
I20260812 02:41:24.261267 30919 raft_consensus.cc:2229] T 5f630de783264afe889a2adf30269a04 P 1462f7a54ea54853a62604e14c997f41 [term 1 LEADER]: Raft consensus shutting down.
I20260812 02:41:24.262235 30919 raft_consensus.cc:2258] T 5f630de783264afe889a2adf30269a04 P 1462f7a54ea54853a62604e14c997f41 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:41:24.265127 30919 ts_tablet_manager.cc:1905] T 5f630de783264afe889a2adf30269a04 P 1462f7a54ea54853a62604e14c997f41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:41:24.265775 30920 tablet_replica.cc:330] T 5f630de783264afe889a2adf30269a04 P 6a24b3e160054e808fb901f47801c4cb: stopping tablet replica
I20260812 02:41:24.266310 30920 raft_consensus.cc:2229] T 5f630de783264afe889a2adf30269a04 P 6a24b3e160054e808fb901f47801c4cb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 02:41:24.266961 30920 raft_consensus.cc:2258] T 5f630de783264afe889a2adf30269a04 P 6a24b3e160054e808fb901f47801c4cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:41:24.269219 30918 tablet_replica.cc:330] T 5f630de783264afe889a2adf30269a04 P 9058cc1382ac496780afcc0e6f51d87a: stopping tablet replica
I20260812 02:41:24.269426 30920 ts_tablet_manager.cc:1905] T 5f630de783264afe889a2adf30269a04 P 6a24b3e160054e808fb901f47801c4cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:41:24.271610 30918 raft_consensus.cc:2229] T 5f630de783264afe889a2adf30269a04 P 9058cc1382ac496780afcc0e6f51d87a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 02:41:24.272783 30918 raft_consensus.cc:2258] T 5f630de783264afe889a2adf30269a04 P 9058cc1382ac496780afcc0e6f51d87a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:41:24.273725 30919 ts_tablet_manager.cc:1918] T 5f630de783264afe889a2adf30269a04 P 1462f7a54ea54853a62604e14c997f41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 02:41:24.273943 30919 log.cc:1196] T 5f630de783264afe889a2adf30269a04 P 1462f7a54ea54853a62604e14c997f41: Deleting WAL directory at /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502156971023-7698-0/minicluster-data/ts-2/wal/wals/5f630de783264afe889a2adf30269a04
I20260812 02:41:24.274586 30919 ts_tablet_manager.cc:1939] T 5f630de783264afe889a2adf30269a04 P 1462f7a54ea54853a62604e14c997f41: Deleting consensus metadata
I20260812 02:41:24.275410 30918 ts_tablet_manager.cc:1905] T 5f630de783264afe889a2adf30269a04 P 9058cc1382ac496780afcc0e6f51d87a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:41:24.276567 30322 catalog_manager.cc:4841] TS 1462f7a54ea54853a62604e14c997f41 (127.7.132.131:36957): tablet 5f630de783264afe889a2adf30269a04 (table test-table [id=935cb65c954d43368f2bf1362b26facb]) successfully deleted
I20260812 02:41:24.278777 30885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:24.247638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58636 (local address 127.7.132.129:40655)
0812 02:41:24.247823 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0812 02:41:24.247830 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:41:24.263090 (+ 15260us) server_negotiation.cc:407] Connection header received
0812 02:41:24.263668 (+ 578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:41:24.263678 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:41:24.263806 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:41:24.263910 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:41:24.264708 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:24.265983 (+ 1275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:24.269515 (+ 3532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:24.270281 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:24.270402 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:24.276634 (+ 6232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:41:24.276646 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:41:24.276666 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:41:24.277559 (+ 893us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:41:24.277633 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:41:24.277666 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:41:24.277772 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0812 02:41:24.278186 (+ 414us) server_negotiation.cc:299] Negotiation successful
0812 02:41:24.278344 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260812 02:41:24.279579 30920 ts_tablet_manager.cc:1918] T 5f630de783264afe889a2adf30269a04 P 6a24b3e160054e808fb901f47801c4cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 02:41:24.279806 30920 log.cc:1196] T 5f630de783264afe889a2adf30269a04 P 6a24b3e160054e808fb901f47801c4cb: Deleting WAL directory at /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786502156971023-7698-0/minicluster-data/ts-1/wal/wals/5f630de783264afe889a2adf30269a04
I20260812 02:41:24.280364 30555 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:41:24.280519 30920 ts_tablet_manager.cc:1939] T 5f630de783264afe889a2adf30269a04 P 6a24b3e160054e808fb901f47801c4cb: Deleting consensus metadata
I20260812 02:41:24.283205 30322 catalog_manager.cc:4841] TS 6a24b3e160054e808fb901f47801c4cb (127.7.132.130:37793): tablet 5f630de783264afe889a2adf30269a04 (table test-table [id=935cb65c954d43368f2bf1362b26facb]) successfully deleted
I20260812 02:41:24.395305 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 30408
I20260812 02:41:24.416798 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 30565
W20260812 02:41:24.416844 30321 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.132.129:40655 (error 108)
I20260812 02:41:24.419401 30924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:24.418410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:40655 (local address 127.0.0.1:58644)
0812 02:41:24.419070 (+ 660us) negotiation.cc:107] Waiting for socket to connect
0812 02:41:24.419228 (+ 158us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.129:40655: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":491,"thread_start_us":356,"threads_started":1}
W20260812 02:41:24.419692 30321 catalog_manager.cc:4584] TS 9058cc1382ac496780afcc0e6f51d87a (127.7.132.129:40655): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5f630de783264afe889a2adf30269a04: Network error: Client connection negotiation failed: client connection to 127.7.132.129:40655: connect: Connection refused (error 111)
I20260812 02:41:24.425136 30890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:24.418060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47956 (local address 127.7.132.130:37793)
0812 02:41:24.418262 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0812 02:41:24.418267 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:41:24.418284 (+ 17us) server_negotiation.cc:407] Connection header received
0812 02:41:24.418340 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:41:24.418344 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:41:24.418432 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:41:24.418550 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:41:24.419069 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:24.420242 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:24.421537 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:24.422321 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:24.422455 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:24.423514 (+ 1059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:41:24.423526 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:41:24.423545 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:41:24.424356 (+ 811us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:41:24.424425 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:41:24.424450 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:41:24.424565 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0812 02:41:24.424657 (+ 92us) server_negotiation.cc:299] Negotiation successful
0812 02:41:24.424766 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105}
I20260812 02:41:24.425874 30713 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:41:24.473330 30924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:24.472721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:40655 (local address 127.0.0.1:58648)
0812 02:41:24.472980 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0812 02:41:24.473095 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.129:40655: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260812 02:41:24.474642 30924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:24.474190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:40655 (local address 127.0.0.1:58664)
0812 02:41:24.474368 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0812 02:41:24.474468 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.129:40655: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260812 02:41:24.517724 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 30565
I20260812 02:41:24.535187 30924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:24.534713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:40655 (local address 127.0.0.1:58672)
0812 02:41:24.534919 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0812 02:41:24.535034 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.129:40655: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260812 02:41:24.536339 30924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:24.535939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:40655 (local address 127.0.0.1:58688)
0812 02:41:24.536093 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0812 02:41:24.536182 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.129:40655: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260812 02:41:24.548058 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 30722
I20260812 02:41:24.557207 30887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:24.549015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59454 (local address 127.7.132.131:36957)
0812 02:41:24.549224 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0812 02:41:24.549231 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:41:24.549261 (+ 30us) server_negotiation.cc:407] Connection header received
0812 02:41:24.549328 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:41:24.549333 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:41:24.549466 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:41:24.549605 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:41:24.550433 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:24.551653 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:24.553387 (+ 1734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:24.554068 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:24.554343 (+ 275us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:24.555493 (+ 1150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:41:24.555512 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:41:24.555532 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:41:24.556393 (+ 861us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:41:24.556462 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:41:24.556490 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:41:24.556608 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0812 02:41:24.556714 (+ 106us) server_negotiation.cc:299] Negotiation successful
0812 02:41:24.556812 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260812 02:41:24.558002 30863 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:41:24.617236 30924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:24.616699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:40655 (local address 127.0.0.1:58690)
0812 02:41:24.616959 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0812 02:41:24.617085 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.129:40655: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
I20260812 02:41:24.618494 30924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:24.617936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:40655 (local address 127.0.0.1:58704)
0812 02:41:24.618206 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0812 02:41:24.618325 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.129:40655: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105}
I20260812 02:41:24.680450 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 30722
I20260812 02:41:24.706800 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 30300
I20260812 02:41:24.708019 30374 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:41:24.820060 30924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:24.819550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:40655 (local address 127.0.0.1:58706)
0812 02:41:24.819777 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0812 02:41:24.819898 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.129:40655: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260812 02:41:24.820091 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 30300
2026-08-12T02:41:24Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (3248 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502156971023-7698-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 02:41:24 dist-test-slave-mn4r krb5kdc[30930](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 02:41:24 dist-test-slave-mn4r krb5kdc[30930](info): set up 2 sockets
Aug 12 02:41:24 dist-test-slave-mn4r krb5kdc[30930](info): commencing operation
krb5kdc: starting...
W20260812 02:41:26.116539 7698 mini_kdc.cc:121] Time spent starting KDC: real 1.250s user 0.001s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502156971023-7698-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502156971023-7698-0/krb5kdc/test-user.keytab.
Aug 12 02:41:26 dist-test-slave-mn4r krb5kdc[30930](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502486, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T02:41:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T02:41:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.132.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
I20260812 02:41:26.441313 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502156971023-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502156971023-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502156971023-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:46677
--webserver_interface=127.7.132.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:42973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.132.190:46677
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:41:26.791399 30946 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:41:26.791745 30946 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:41:26.791842 30946 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:41:26.799058 30946 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:41:26.799182 30946 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:41:26.799229 30946 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:41:26.799288 30946 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:41:26.799330 30946 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:41:26.809348 30946 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:42973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502156971023-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502156971023-7698-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.132.190:46677
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:46677
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.132.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502156971023-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:41:26.810760 30946 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:41:26.812165 30946 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:41:26.824471 30951 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:41:26.824391 30952 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:41:26.824438 30954 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:41:26.825277 30946 server_base.cc:1034] running on GCE node
I20260812 02:41:26.826164 30946 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:41:26.827632 30946 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:41:26.828840 30946 hybrid_clock.cc:648] HybridClock initialized: now 1786502486828781 us; error 53 us; skew 500 ppm
I20260812 02:41:26.829221 30946 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:41:26 dist-test-slave-mn4r krb5kdc[30930](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502486, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:41:26.832512 30946 init.cc:377] Logged in from keytab as kudu/127.7.132.190@KRBTEST.COM (short username kudu)
I20260812 02:41:26.834009 30946 webserver.cc:460] Webserver started at http://127.7.132.190:36531/ using document root <none> and password file <none>
I20260812 02:41:26.834755 30946 fs_manager.cc:359] Metadata directory not provided
I20260812 02:41:26.834865 30946 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:41:26.835141 30946 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:41:26.838235 30946 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502156971023-7698-0/minicluster-data/master-0/data/instance:
uuid: "9a9539077c254ad48d0c621da3924a34"
format_stamp: "Formatted at 2026-08-12 02:41:26 on dist-test-slave-mn4r"
server_key: "ed3c01c874c234414a7b056ed2be0379"
server_key_iv: "f4b720e85147bd6d193fcf1248da960b"
server_key_version: "encryptionkey@0"
I20260812 02:41:26.838968 30946 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502156971023-7698-0/minicluster-data/master-0/wal/instance:
uuid: "9a9539077c254ad48d0c621da3924a34"
format_stamp: "Formatted at 2026-08-12 02:41:26 on dist-test-slave-mn4r"
server_key: "ed3c01c874c234414a7b056ed2be0379"
server_key_iv: "f4b720e85147bd6d193fcf1248da960b"
server_key_version: "encryptionkey@0"
I20260812 02:41:26.844249 30946 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 02:41:26.847728 30962 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:41:26.849049 30946 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 02:41:26.849248 30946 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502156971023-7698-0/minicluster-data/master-0/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502156971023-7698-0/minicluster-data/master-0/wal
uuid: "9a9539077c254ad48d0c621da3924a34"
format_stamp: "Formatted at 2026-08-12 02:41:26 on dist-test-slave-mn4r"
server_key: "ed3c01c874c234414a7b056ed2be0379"
server_key_iv: "f4b720e85147bd6d193fcf1248da960b"
server_key_version: "encryptionkey@0"
I20260812 02:41:26.849442 30946 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502156971023-7698-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502156971023-7698-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502156971023-7698-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:41:26.867316 30946 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:41:26.871971 30946 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:41:26.872285 30946 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:41:26.891279 30946 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.190:46677
I20260812 02:41:26.891378 31025 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.190:46677 every 8 connection(s)
I20260812 02:41:26.892746 30946 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
I20260812 02:41:26.893453 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 30946
I20260812 02:41:26.893666 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502156971023-7698-0/minicluster-data/master-0/wal/instance
I20260812 02:41:26.894300 7698 external_mini_cluster.cc:1428] Setting key c7162be25ee81e6b60512f44f8942953
I20260812 02:41:26.897413 31026 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 12 02:41:26 dist-test-slave-mn4r krb5kdc[30930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502486, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 02:41:26.911231 31026 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a9539077c254ad48d0c621da3924a34: Bootstrap starting.
I20260812 02:41:26.915007 31026 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9a9539077c254ad48d0c621da3924a34: Neither blocks nor log segments found. Creating new log.
I20260812 02:41:26.916055 31026 log.cc:824] T 00000000000000000000000000000000 P 9a9539077c254ad48d0c621da3924a34: Log is configured to *not* fsync() on all Append() calls
I20260812 02:41:26.919055 31029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:26.896306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56240 (local address 127.7.132.190:46677)
0812 02:41:26.897114 (+ 808us) server_negotiation.cc:206] Beginning negotiation
0812 02:41:26.897127 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 02:41:26.897168 (+ 41us) server_negotiation.cc:407] Connection header received
0812 02:41:26.898898 (+ 1730us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:41:26.898921 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:41:26.899304 (+ 383us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:41:26.899616 (+ 312us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:41:26.901522 (+ 1906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:26.903001 (+ 1479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:26.904934 (+ 1933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:26.905620 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:26.905748 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:26.911037 (+ 5289us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:41:26.911046 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:41:26.911061 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:41:26.911101 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:41:26.914789 (+ 3688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:41:26.915614 (+ 825us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:41:26.915627 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:41:26.915634 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:41:26.915805 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:41:26.916234 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:41:26.916241 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:41:26.916245 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:41:26.916770 (+ 525us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:41:26.917135 (+ 365us) server_negotiation.cc:1035] Waiting for connection context
0812 02:41:26.917579 (+ 444us) server_negotiation.cc:299] Negotiation successful
0812 02:41:26.918024 (+ 445us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":567,"thread_start_us":354,"threads_started":1}
I20260812 02:41:26.919237 31026 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a9539077c254ad48d0c621da3924a34: No bootstrap required, opened a new log
I20260812 02:41:26.927373 31026 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9a9539077c254ad48d0c621da3924a34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a9539077c254ad48d0c621da3924a34" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 46677 } }
I20260812 02:41:26.927862 31026 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9a9539077c254ad48d0c621da3924a34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:41:26.928025 31026 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9a9539077c254ad48d0c621da3924a34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a9539077c254ad48d0c621da3924a34, State: Initialized, Role: FOLLOWER
I20260812 02:41:26.928742 31026 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9a9539077c254ad48d0c621da3924a34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a9539077c254ad48d0c621da3924a34" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 46677 } }
I20260812 02:41:26.929015 31026 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9a9539077c254ad48d0c621da3924a34 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 02:41:26.929175 31026 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9a9539077c254ad48d0c621da3924a34 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 02:41:26.929368 31026 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9a9539077c254ad48d0c621da3924a34 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:41:26.931679 31026 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9a9539077c254ad48d0c621da3924a34 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a9539077c254ad48d0c621da3924a34" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 46677 } }
I20260812 02:41:26.932176 31026 leader_election.cc:302] T 00000000000000000000000000000000 P 9a9539077c254ad48d0c621da3924a34 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9a9539077c254ad48d0c621da3924a34; no voters:
I20260812 02:41:26.932775 31026 leader_election.cc:288] T 00000000000000000000000000000000 P 9a9539077c254ad48d0c621da3924a34 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 02:41:26.932910 31031 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9a9539077c254ad48d0c621da3924a34 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:41:26.933660 31031 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9a9539077c254ad48d0c621da3924a34 [term 1 LEADER]: Becoming Leader. State: Replica: 9a9539077c254ad48d0c621da3924a34, State: Running, Role: LEADER
I20260812 02:41:26.934322 31031 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9a9539077c254ad48d0c621da3924a34 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a9539077c254ad48d0c621da3924a34" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 46677 } }
I20260812 02:41:26.935199 31026 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9a9539077c254ad48d0c621da3924a34 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:41:26.937695 31032 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a9539077c254ad48d0c621da3924a34 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9a9539077c254ad48d0c621da3924a34" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a9539077c254ad48d0c621da3924a34" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 46677 } } }
I20260812 02:41:26.937683 31033 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a9539077c254ad48d0c621da3924a34 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9a9539077c254ad48d0c621da3924a34. Latest consensus state: current_term: 1 leader_uuid: "9a9539077c254ad48d0c621da3924a34" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a9539077c254ad48d0c621da3924a34" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 46677 } } }
I20260812 02:41:26.938167 31032 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a9539077c254ad48d0c621da3924a34 [sys.catalog]: This master's current role is: LEADER
I20260812 02:41:26.938226 31033 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a9539077c254ad48d0c621da3924a34 [sys.catalog]: This master's current role is: LEADER
I20260812 02:41:26.940867 31039 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:41:26.946569 31039 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 02:41:26.955528 31039 catalog_manager.cc:1347] Generated new cluster ID: 08e43edaefe84bb39539cfef6ac67427
I20260812 02:41:26.955642 31039 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 02:41:26.965871 31039 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 02:41:26.967768 31039 catalog_manager.cc:1504] Loading token signing keys...
I20260812 02:41:26.978822 31039 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9a9539077c254ad48d0c621da3924a34: Generated new TSK 0
I20260812 02:41:26.979879 31039 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.132.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.129.keytab.
Entry for principal kudu/127.7.132.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.129.keytab.
I20260812 02:41:27.055363 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502156971023-7698-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502156971023-7698-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502156971023-7698-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502156971023-7698-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.129:0
--local_ip_for_outbound_sockets=127.7.132.129
--webserver_interface=127.7.132.129
--webserver_port=0
--tserver_master_addrs=127.7.132.190:46677
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.129.keytab
--principal=kudu/127.7.132.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:42973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:41:27.412142 31055 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:41:27.412433 31055 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:41:27.412644 31055 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:41:27.420374 31055 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:41:27.420488 31055 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:41:27.420703 31055 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.129
I20260812 02:41:27.430820 31055 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:42973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502156971023-7698-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502156971023-7698-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.129.keytab
--principal=kudu/127.7.132.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502156971023-7698-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.132.129
--webserver_port=0
--tserver_master_addrs=127.7.132.190:46677
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.129
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502156971023-7698-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:41:27.432250 31055 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:41:27.433655 31055 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:41:27.447690 31060 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:41:27.447722 31062 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:41:27.447880 31064 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:41:27.448635 31055 server_base.cc:1034] running on GCE node
I20260812 02:41:27.449297 31055 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:41:27.450301 31055 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:41:27.451637 31055 hybrid_clock.cc:648] HybridClock initialized: now 1786502487451554 us; error 60 us; skew 500 ppm
I20260812 02:41:27.451997 31055 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:41:27 dist-test-slave-mn4r krb5kdc[30930](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502487, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:41:27.455395 31055 init.cc:377] Logged in from keytab as kudu/127.7.132.129@KRBTEST.COM (short username kudu)
I20260812 02:41:27.456897 31055 webserver.cc:460] Webserver started at http://127.7.132.129:38927/ using document root <none> and password file <none>
I20260812 02:41:27.457588 31055 fs_manager.cc:359] Metadata directory not provided
I20260812 02:41:27.457701 31055 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:41:27.457974 31055 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:41:27.461074 31055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502156971023-7698-0/minicluster-data/ts-0/data/instance:
uuid: "dedc1d8791af46b08b7f56ba65052e15"
format_stamp: "Formatted at 2026-08-12 02:41:27 on dist-test-slave-mn4r"
server_key: "b1da076b6eed9f26ea0548d3674a17e7"
server_key_iv: "79b159af433ff86a9ef36e6da951799a"
server_key_version: "encryptionkey@0"
I20260812 02:41:27.461860 31055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502156971023-7698-0/minicluster-data/ts-0/wal/instance:
uuid: "dedc1d8791af46b08b7f56ba65052e15"
format_stamp: "Formatted at 2026-08-12 02:41:27 on dist-test-slave-mn4r"
server_key: "b1da076b6eed9f26ea0548d3674a17e7"
server_key_iv: "79b159af433ff86a9ef36e6da951799a"
server_key_version: "encryptionkey@0"
I20260812 02:41:27.467124 31055 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260812 02:41:27.470577 31074 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:41:27.471917 31055 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260812 02:41:27.472047 31055 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502156971023-7698-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502156971023-7698-0/minicluster-data/ts-0/wal
uuid: "dedc1d8791af46b08b7f56ba65052e15"
format_stamp: "Formatted at 2026-08-12 02:41:27 on dist-test-slave-mn4r"
server_key: "b1da076b6eed9f26ea0548d3674a17e7"
server_key_iv: "79b159af433ff86a9ef36e6da951799a"
server_key_version: "encryptionkey@0"
I20260812 02:41:27.472218 31055 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502156971023-7698-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502156971023-7698-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502156971023-7698-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:41:27.488469 31055 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:41:27.492415 31055 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:41:27.492667 31055 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:41:27.493620 31055 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:41:27.495040 31055 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:41:27.495122 31055 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:41:27.495206 31055 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:41:27.495254 31055 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:41:27.529574 31055 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.129:43495
I20260812 02:41:27.529630 31205 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.129:43495 every 8 connection(s)
I20260812 02:41:27.530978 31055 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502156971023-7698-0/minicluster-data/ts-0/data/info.pb
I20260812 02:41:27.538717 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 31055
I20260812 02:41:27.538910 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502156971023-7698-0/minicluster-data/ts-0/wal/instance
I20260812 02:41:27.539389 7698 external_mini_cluster.cc:1428] Setting key 9bf02d4144c7b50cc02f62f94d603dcd
Aug 12 02:41:27 dist-test-slave-mn4r krb5kdc[30930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502487, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.129@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 02:41:27.553988 31209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:27.533629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:54483 (local address 127.7.132.190:46677)
0812 02:41:27.534149 (+ 520us) server_negotiation.cc:206] Beginning negotiation
0812 02:41:27.534154 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:41:27.535209 (+ 1055us) server_negotiation.cc:407] Connection header received
0812 02:41:27.537217 (+ 2008us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:41:27.537223 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:41:27.537334 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:41:27.537460 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:41:27.539201 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:27.540159 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:27.542264 (+ 2105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:27.542973 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:27.543136 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:27.546819 (+ 3683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:41:27.546828 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:41:27.546835 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:41:27.546880 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:41:27.550509 (+ 3629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:41:27.551257 (+ 748us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:41:27.551264 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:41:27.551271 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:41:27.551429 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:41:27.551851 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:41:27.551858 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:41:27.551864 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:41:27.552273 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:41:27.552454 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0812 02:41:27.553220 (+ 766us) server_negotiation.cc:299] Negotiation successful
0812 02:41:27.553511 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":232,"threads_started":1}
I20260812 02:41:27.554806 31210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:27.534302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:46677 (local address 127.7.132.129:54483)
0812 02:41:27.535040 (+ 738us) negotiation.cc:107] Waiting for socket to connect
0812 02:41:27.535085 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 02:41:27.536940 (+ 1855us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:41:27.537729 (+ 789us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:41:27.537737 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:41:27.538333 (+ 596us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:41:27.538981 (+ 648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:41:27.539031 (+ 50us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:41:27.540336 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:41:27.540340 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:41:27.542046 (+ 1706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:41:27.542059 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:41:27.543213 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:41:27.543218 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:41:27.543418 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:27.544132 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:41:27.544155 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:41:27.546577 (+ 2422us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:41:27.550658 (+ 4081us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:41:27.550665 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:41:27.550677 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:41:27.551102 (+ 425us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:41:27.551538 (+ 436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:41:27.551541 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:41:27.551543 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:41:27.551732 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:41:27.552439 (+ 707us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:41:27.552445 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:41:27.552775 (+ 330us) client_negotiation.cc:769] Sending connection context
0812 02:41:27.553011 (+ 236us) client_negotiation.cc:240] Negotiation successful
0812 02:41:27.553584 (+ 573us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":270,"threads_started":1}
I20260812 02:41:27.556455 31207 heartbeater.cc:347] Connected to a master server at 127.7.132.190:46677
I20260812 02:41:27.556815 31207 heartbeater.cc:464] Registering TS with master...
I20260812 02:41:27.557569 31207 heartbeater.cc:511] Master 127.7.132.190:46677 requested a full tablet report, sending...
I20260812 02:41:27.559901 30984 ts_manager.cc:194] Registered new tserver with Master: dedc1d8791af46b08b7f56ba65052e15 (127.7.132.129:43495)
I20260812 02:41:27.561336 30984 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.129@KRBTEST.COM'} at 127.7.132.129:54483
WARNING: no policy specified for kudu/127.7.132.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.130.keytab.
Entry for principal kudu/127.7.132.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.130.keytab.
I20260812 02:41:27.609714 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502156971023-7698-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502156971023-7698-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502156971023-7698-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502156971023-7698-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.130:0
--local_ip_for_outbound_sockets=127.7.132.130
--webserver_interface=127.7.132.130
--webserver_port=0
--tserver_master_addrs=127.7.132.190:46677
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.130.keytab
--principal=kudu/127.7.132.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:42973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:41:27.957291 31213 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:41:27.957576 31213 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:41:27.957669 31213 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:41:27.964891 31213 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:41:27.964998 31213 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:41:27.965122 31213 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.130
I20260812 02:41:27.974743 31213 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:42973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502156971023-7698-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502156971023-7698-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.130.keytab
--principal=kudu/127.7.132.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502156971023-7698-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.132.130
--webserver_port=0
--tserver_master_addrs=127.7.132.190:46677
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.130
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502156971023-7698-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:41:27.976142 31213 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:41:27.977528 31213 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:41:27.990947 31219 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:41:27.990906 31221 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:41:27.991029 31218 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:41:27.991860 31213 server_base.cc:1034] running on GCE node
I20260812 02:41:27.992493 31213 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:41:27.993491 31213 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:41:27.994714 31213 hybrid_clock.cc:648] HybridClock initialized: now 1786502487994668 us; error 40 us; skew 500 ppm
I20260812 02:41:27.995049 31213 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:41:27 dist-test-slave-mn4r krb5kdc[30930](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502487, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:41:27.998354 31213 init.cc:377] Logged in from keytab as kudu/127.7.132.130@KRBTEST.COM (short username kudu)
I20260812 02:41:27.999871 31213 webserver.cc:460] Webserver started at http://127.7.132.130:42853/ using document root <none> and password file <none>
I20260812 02:41:28.000589 31213 fs_manager.cc:359] Metadata directory not provided
I20260812 02:41:28.000744 31213 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:41:28.001077 31213 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:41:28.004123 31213 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502156971023-7698-0/minicluster-data/ts-1/data/instance:
uuid: "734e844b983d4bc0a5ce44ce146295b8"
format_stamp: "Formatted at 2026-08-12 02:41:27 on dist-test-slave-mn4r"
server_key: "6b2aa0f5ec9cbb029bb18672426c7211"
server_key_iv: "195d5dd80e87153818abd057084a0a17"
server_key_version: "encryptionkey@0"
I20260812 02:41:28.004948 31213 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502156971023-7698-0/minicluster-data/ts-1/wal/instance:
uuid: "734e844b983d4bc0a5ce44ce146295b8"
format_stamp: "Formatted at 2026-08-12 02:41:27 on dist-test-slave-mn4r"
server_key: "6b2aa0f5ec9cbb029bb18672426c7211"
server_key_iv: "195d5dd80e87153818abd057084a0a17"
server_key_version: "encryptionkey@0"
I20260812 02:41:28.010272 31213 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260812 02:41:28.013646 31230 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:41:28.015023 31213 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 02:41:28.015216 31213 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502156971023-7698-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502156971023-7698-0/minicluster-data/ts-1/wal
uuid: "734e844b983d4bc0a5ce44ce146295b8"
format_stamp: "Formatted at 2026-08-12 02:41:27 on dist-test-slave-mn4r"
server_key: "6b2aa0f5ec9cbb029bb18672426c7211"
server_key_iv: "195d5dd80e87153818abd057084a0a17"
server_key_version: "encryptionkey@0"
I20260812 02:41:28.015434 31213 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502156971023-7698-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502156971023-7698-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502156971023-7698-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:41:28.036767 31213 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:41:28.041476 31213 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:41:28.041783 31213 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:41:28.042856 31213 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:41:28.044186 31213 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:41:28.044298 31213 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:41:28.044452 31213 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:41:28.044525 31213 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:41:28.078006 31213 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.130:32877
I20260812 02:41:28.078100 31368 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.130:32877 every 8 connection(s)
I20260812 02:41:28.079628 31213 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502156971023-7698-0/minicluster-data/ts-1/data/info.pb
I20260812 02:41:28.080790 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 31213
I20260812 02:41:28.080993 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502156971023-7698-0/minicluster-data/ts-1/wal/instance
I20260812 02:41:28.081539 7698 external_mini_cluster.cc:1428] Setting key 41008adfc6b69128b19bac586846583b
Aug 12 02:41:28 dist-test-slave-mn4r krb5kdc[30930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502487, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.130@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 02:41:28.105136 31371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:28.082688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:42745 (local address 127.7.132.190:46677)
0812 02:41:28.083198 (+ 510us) server_negotiation.cc:206] Beginning negotiation
0812 02:41:28.083205 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:41:28.084440 (+ 1235us) server_negotiation.cc:407] Connection header received
0812 02:41:28.086633 (+ 2193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:41:28.086642 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:41:28.086735 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:41:28.086854 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:41:28.089212 (+ 2358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:28.090179 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:28.092733 (+ 2554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:28.093325 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:28.093493 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:28.098533 (+ 5040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:41:28.098539 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:41:28.098542 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:41:28.098579 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:41:28.101570 (+ 2991us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:41:28.102511 (+ 941us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:41:28.102523 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:41:28.102528 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:41:28.102675 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:41:28.103178 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:41:28.103182 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:41:28.103184 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:41:28.103468 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:41:28.103648 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0812 02:41:28.104460 (+ 812us) server_negotiation.cc:299] Negotiation successful
0812 02:41:28.104723 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":390,"thread_start_us":258,"threads_started":1}
I20260812 02:41:28.106333 31372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:28.083339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:46677 (local address 127.7.132.130:42745)
0812 02:41:28.084240 (+ 901us) negotiation.cc:107] Waiting for socket to connect
0812 02:41:28.084297 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0812 02:41:28.086336 (+ 2039us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:41:28.087324 (+ 988us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:41:28.087338 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:41:28.088160 (+ 822us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:41:28.088986 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:41:28.089015 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:41:28.090346 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:41:28.090354 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:41:28.092553 (+ 2199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:41:28.092573 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:41:28.093600 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:41:28.093609 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:41:28.093943 (+ 334us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:28.094824 (+ 881us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:41:28.094859 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:41:28.098240 (+ 3381us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:41:28.101746 (+ 3506us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:41:28.101756 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:41:28.101775 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:41:28.102368 (+ 593us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:41:28.102802 (+ 434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:41:28.102808 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:41:28.102812 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:41:28.103033 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:41:28.103618 (+ 585us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:41:28.103627 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:41:28.104020 (+ 393us) client_negotiation.cc:769] Sending connection context
0812 02:41:28.104334 (+ 314us) client_negotiation.cc:240] Negotiation successful
0812 02:41:28.104934 (+ 600us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":600,"thread_start_us":378,"threads_started":1}
I20260812 02:41:28.108065 31369 heartbeater.cc:347] Connected to a master server at 127.7.132.190:46677
I20260812 02:41:28.108429 31369 heartbeater.cc:464] Registering TS with master...
I20260812 02:41:28.109212 31369 heartbeater.cc:511] Master 127.7.132.190:46677 requested a full tablet report, sending...
I20260812 02:41:28.110994 30984 ts_manager.cc:194] Registered new tserver with Master: 734e844b983d4bc0a5ce44ce146295b8 (127.7.132.130:32877)
I20260812 02:41:28.112368 30984 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.130@KRBTEST.COM'} at 127.7.132.130:42745
WARNING: no policy specified for kudu/127.7.132.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.131.keytab.
Entry for principal kudu/127.7.132.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.131.keytab.
I20260812 02:41:28.153024 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502156971023-7698-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502156971023-7698-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502156971023-7698-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502156971023-7698-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.131:0
--local_ip_for_outbound_sockets=127.7.132.131
--webserver_interface=127.7.132.131
--webserver_port=0
--tserver_master_addrs=127.7.132.190:46677
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.131.keytab
--principal=kudu/127.7.132.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:42973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:41:28.499035 31375 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:41:28.499346 31375 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:41:28.499444 31375 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:41:28.506882 31375 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:41:28.507005 31375 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:41:28.507123 31375 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.131
I20260812 02:41:28.517091 31375 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:42973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502156971023-7698-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502156971023-7698-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.131.keytab
--principal=kudu/127.7.132.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502156971023-7698-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.132.131
--webserver_port=0
--tserver_master_addrs=127.7.132.190:46677
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.131
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502156971023-7698-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:41:28.518447 31375 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:41:28.519920 31375 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:41:28.534710 31381 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:41:28.534684 31382 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:41:28.535481 31384 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:41:28.536075 31375 server_base.cc:1034] running on GCE node
I20260812 02:41:28.536736 31375 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:41:28.537678 31375 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:41:28.538874 31375 hybrid_clock.cc:648] HybridClock initialized: now 1786502488538819 us; error 39 us; skew 500 ppm
I20260812 02:41:28.539220 31375 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:41:28 dist-test-slave-mn4r krb5kdc[30930](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502488, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:41:28.542487 31375 init.cc:377] Logged in from keytab as kudu/127.7.132.131@KRBTEST.COM (short username kudu)
I20260812 02:41:28.543915 31375 webserver.cc:460] Webserver started at http://127.7.132.131:34293/ using document root <none> and password file <none>
I20260812 02:41:28.544550 31375 fs_manager.cc:359] Metadata directory not provided
I20260812 02:41:28.544661 31375 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:41:28.544929 31375 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:41:28.547858 31375 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502156971023-7698-0/minicluster-data/ts-2/data/instance:
uuid: "f6c631bbe8ff4808a248d32f53ebd11b"
format_stamp: "Formatted at 2026-08-12 02:41:28 on dist-test-slave-mn4r"
server_key: "1458d5c38660497f486c4deaeb82ea1b"
server_key_iv: "237544f852ea19039f0b9e37c60c03a9"
server_key_version: "encryptionkey@0"
I20260812 02:41:28.548580 31375 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502156971023-7698-0/minicluster-data/ts-2/wal/instance:
uuid: "f6c631bbe8ff4808a248d32f53ebd11b"
format_stamp: "Formatted at 2026-08-12 02:41:28 on dist-test-slave-mn4r"
server_key: "1458d5c38660497f486c4deaeb82ea1b"
server_key_iv: "237544f852ea19039f0b9e37c60c03a9"
server_key_version: "encryptionkey@0"
I20260812 02:41:28.553685 31375 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260812 02:41:28.557112 31392 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:41:28.558506 31375 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 02:41:28.558642 31375 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502156971023-7698-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502156971023-7698-0/minicluster-data/ts-2/wal
uuid: "f6c631bbe8ff4808a248d32f53ebd11b"
format_stamp: "Formatted at 2026-08-12 02:41:28 on dist-test-slave-mn4r"
server_key: "1458d5c38660497f486c4deaeb82ea1b"
server_key_iv: "237544f852ea19039f0b9e37c60c03a9"
server_key_version: "encryptionkey@0"
I20260812 02:41:28.558801 31375 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502156971023-7698-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502156971023-7698-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502156971023-7698-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:41:28.565230 31207 heartbeater.cc:503] Master 127.7.132.190:46677 was elected leader, sending a full tablet report...
I20260812 02:41:28.575479 31375 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:41:28.579677 31375 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:41:28.579969 31375 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:41:28.580914 31375 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:41:28.582373 31375 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:41:28.582474 31375 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:41:28.582626 31375 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:41:28.582705 31375 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:41:28.616523 31375 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.131:33485
I20260812 02:41:28.616575 31520 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.131:33485 every 8 connection(s)
I20260812 02:41:28.617880 31375 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502156971023-7698-0/minicluster-data/ts-2/data/info.pb
I20260812 02:41:28.624559 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 31375
I20260812 02:41:28.624763 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502156971023-7698-0/minicluster-data/ts-2/wal/instance
I20260812 02:41:28.625337 7698 external_mini_cluster.cc:1428] Setting key 3e72ffe9ac4a6355624667c0c1a8c031
Aug 12 02:41:28 dist-test-slave-mn4r krb5kdc[30930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502488, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.131@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 02:41:28.638947 31523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:28.620618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:42415 (local address 127.7.132.190:46677)
0812 02:41:28.621079 (+ 461us) server_negotiation.cc:206] Beginning negotiation
0812 02:41:28.621084 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:41:28.622128 (+ 1044us) server_negotiation.cc:407] Connection header received
0812 02:41:28.623999 (+ 1871us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:41:28.624003 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:41:28.624071 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:41:28.624192 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:41:28.626277 (+ 2085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:28.627287 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:28.629235 (+ 1948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:28.629630 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:28.629726 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:28.633133 (+ 3407us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:41:28.633138 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:41:28.633142 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:41:28.633174 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:41:28.635745 (+ 2571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:41:28.636502 (+ 757us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:41:28.636512 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:41:28.636515 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:41:28.636648 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:41:28.637103 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:41:28.637112 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:41:28.637114 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:41:28.637393 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:41:28.637563 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0812 02:41:28.638288 (+ 725us) server_negotiation.cc:299] Negotiation successful
0812 02:41:28.638521 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":220,"threads_started":1}
I20260812 02:41:28.640340 31524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:28.621200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:46677 (local address 127.7.132.131:42415)
0812 02:41:28.621903 (+ 703us) negotiation.cc:107] Waiting for socket to connect
0812 02:41:28.621970 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0812 02:41:28.623742 (+ 1772us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:41:28.624492 (+ 750us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:41:28.624503 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:41:28.625288 (+ 785us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:41:28.626043 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:41:28.626064 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:41:28.627482 (+ 1418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:41:28.627487 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:41:28.629109 (+ 1622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:41:28.629122 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:41:28.629819 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:41:28.629824 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:41:28.630019 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:28.630707 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:41:28.630735 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:41:28.632896 (+ 2161us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:41:28.635885 (+ 2989us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:41:28.635895 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:41:28.635912 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:41:28.636345 (+ 433us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:41:28.636764 (+ 419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:41:28.636769 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:41:28.636773 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:41:28.636971 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:41:28.637530 (+ 559us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:41:28.637538 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:41:28.637893 (+ 355us) client_negotiation.cc:769] Sending connection context
0812 02:41:28.638199 (+ 306us) client_negotiation.cc:240] Negotiation successful
0812 02:41:28.638764 (+ 565us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":278,"threads_started":1}
I20260812 02:41:28.641841 31521 heartbeater.cc:347] Connected to a master server at 127.7.132.190:46677
I20260812 02:41:28.642268 31521 heartbeater.cc:464] Registering TS with master...
I20260812 02:41:28.643028 31521 heartbeater.cc:511] Master 127.7.132.190:46677 requested a full tablet report, sending...
I20260812 02:41:28.644469 30984 ts_manager.cc:194] Registered new tserver with Master: f6c631bbe8ff4808a248d32f53ebd11b (127.7.132.131:33485)
I20260812 02:41:28.645244 30984 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.131@KRBTEST.COM'} at 127.7.132.131:42415
I20260812 02:41:28.653330 7698 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 02:41:28.669965 31523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:28.657423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56256 (local address 127.7.132.190:46677)
0812 02:41:28.657660 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0812 02:41:28.657670 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 02:41:28.658028 (+ 358us) server_negotiation.cc:407] Connection header received
0812 02:41:28.658232 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:41:28.658236 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:41:28.658297 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:41:28.658376 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:41:28.659804 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:28.660511 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:28.661867 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:28.662422 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:28.662514 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:28.664930 (+ 2416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:41:28.664935 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:41:28.664938 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:41:28.664969 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:41:28.667605 (+ 2636us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:41:28.668236 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:41:28.668239 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:41:28.668242 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:41:28.668362 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:41:28.668683 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:41:28.668686 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:41:28.668688 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:41:28.668986 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:41:28.669162 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0812 02:41:28.669452 (+ 290us) server_negotiation.cc:299] Negotiation successful
0812 02:41:28.669618 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260812 02:41:28.673800 30984 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56256:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 02:41:28.680843 30984 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 02:41:28.704839 31537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:28.693290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:43495 (local address 127.0.0.1:43726)
0812 02:41:28.694933 (+ 1643us) negotiation.cc:107] Waiting for socket to connect
0812 02:41:28.694950 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 02:41:28.695053 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:41:28.696955 (+ 1902us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:41:28.696966 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:41:28.697032 (+ 66us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:41:28.697555 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:41:28.697572 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:41:28.699412 (+ 1840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:41:28.699420 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:41:28.701948 (+ 2528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:41:28.701964 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:41:28.703823 (+ 1859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:41:28.703829 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:41:28.704011 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:28.704069 (+ 58us) client_negotiation.cc:769] Sending connection context
0812 02:41:28.704224 (+ 155us) client_negotiation.cc:240] Negotiation successful
0812 02:41:28.704426 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1531,"thread_start_us":1376,"threads_started":1}
I20260812 02:41:28.706061 31541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:28.693964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43726 (local address 127.7.132.129:43495)
0812 02:41:28.696132 (+ 2168us) server_negotiation.cc:206] Beginning negotiation
0812 02:41:28.696145 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 02:41:28.696184 (+ 39us) server_negotiation.cc:407] Connection header received
0812 02:41:28.696307 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:41:28.696319 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:41:28.696663 (+ 344us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:41:28.696910 (+ 247us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:41:28.697813 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:28.699265 (+ 1452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:28.702144 (+ 2879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:28.703618 (+ 1474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:28.703783 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:28.704024 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0812 02:41:28.705672 (+ 1648us) server_negotiation.cc:299] Negotiation successful
0812 02:41:28.705823 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1964,"thread_start_us":1792,"threads_started":1}
I20260812 02:41:28.706169 31539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:28.693363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.130:32877 (local address 127.0.0.1:40678)
0812 02:41:28.697961 (+ 4598us) negotiation.cc:107] Waiting for socket to connect
0812 02:41:28.697981 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 02:41:28.698132 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:41:28.699070 (+ 938us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:41:28.699077 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:41:28.699106 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:41:28.699522 (+ 416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:41:28.699539 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:41:28.701500 (+ 1961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:41:28.701505 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:41:28.703154 (+ 1649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:41:28.703167 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:41:28.705369 (+ 2202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:41:28.705376 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:41:28.705558 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:28.705591 (+ 33us) client_negotiation.cc:769] Sending connection context
0812 02:41:28.705684 (+ 93us) client_negotiation.cc:240] Negotiation successful
0812 02:41:28.705809 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4489,"thread_start_us":4343,"threads_started":1}
I20260812 02:41:28.706707 31542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:28.694038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40678 (local address 127.7.132.130:32877)
0812 02:41:28.695678 (+ 1640us) server_negotiation.cc:206] Beginning negotiation
0812 02:41:28.695691 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 02:41:28.698288 (+ 2597us) server_negotiation.cc:407] Connection header received
0812 02:41:28.698385 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:41:28.698391 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:41:28.698607 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:41:28.698809 (+ 202us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:41:28.699718 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:28.701324 (+ 1606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:28.703318 (+ 1994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:28.704871 (+ 1553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:28.705008 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:28.705207 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0812 02:41:28.706231 (+ 1024us) server_negotiation.cc:299] Negotiation successful
0812 02:41:28.706403 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1457,"thread_start_us":1304,"threads_started":1}
I20260812 02:41:28.707257 31535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:28.693167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.131:33485 (local address 127.0.0.1:41574)
0812 02:41:28.693974 (+ 807us) negotiation.cc:107] Waiting for socket to connect
0812 02:41:28.694025 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 02:41:28.694278 (+ 253us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:41:28.698344 (+ 4066us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:41:28.698348 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:41:28.698372 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:41:28.698814 (+ 442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:41:28.698829 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:41:28.701013 (+ 2184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:41:28.701020 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:41:28.703385 (+ 2365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:41:28.703400 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:41:28.706555 (+ 3155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:41:28.706562 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:41:28.706743 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:28.706780 (+ 37us) client_negotiation.cc:769] Sending connection context
0812 02:41:28.706848 (+ 68us) client_negotiation.cc:240] Negotiation successful
0812 02:41:28.706960 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":500,"thread_start_us":1465,"threads_started":1}
I20260812 02:41:28.707819 31540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:28.693503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41574 (local address 127.7.132.131:33485)
0812 02:41:28.697700 (+ 4197us) server_negotiation.cc:206] Beginning negotiation
0812 02:41:28.697710 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 02:41:28.697748 (+ 38us) server_negotiation.cc:407] Connection header received
0812 02:41:28.697857 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:41:28.697864 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:41:28.698124 (+ 260us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:41:28.699030 (+ 906us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:41:28.699306 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:28.700852 (+ 1546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:28.703551 (+ 2699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:28.706274 (+ 2723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:28.706999 (+ 725us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:28.707249 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0812 02:41:28.707322 (+ 73us) server_negotiation.cc:299] Negotiation successful
0812 02:41:28.707471 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4004,"thread_start_us":3856,"threads_started":1}
I20260812 02:41:28.710553 31127 tablet_service.cc:1467] Processing CreateTablet for tablet e25edded890e44d1b13b479eb65e8c89 (DEFAULT_TABLE table=test-table [id=e8fc0b59ff414c4b9221c94774f9e3d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:41:28.710546 31292 tablet_service.cc:1467] Processing CreateTablet for tablet e25edded890e44d1b13b479eb65e8c89 (DEFAULT_TABLE table=test-table [id=e8fc0b59ff414c4b9221c94774f9e3d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:41:28.711021 31448 tablet_service.cc:1467] Processing CreateTablet for tablet e25edded890e44d1b13b479eb65e8c89 (DEFAULT_TABLE table=test-table [id=e8fc0b59ff414c4b9221c94774f9e3d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:41:28.712074 31127 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e25edded890e44d1b13b479eb65e8c89. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:41:28.712091 31292 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e25edded890e44d1b13b479eb65e8c89. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:41:28.712294 31448 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e25edded890e44d1b13b479eb65e8c89. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:41:28.723860 31543 tablet_bootstrap.cc:492] T e25edded890e44d1b13b479eb65e8c89 P f6c631bbe8ff4808a248d32f53ebd11b: Bootstrap starting.
I20260812 02:41:28.726716 31544 tablet_bootstrap.cc:492] T e25edded890e44d1b13b479eb65e8c89 P dedc1d8791af46b08b7f56ba65052e15: Bootstrap starting.
I20260812 02:41:28.728348 31545 tablet_bootstrap.cc:492] T e25edded890e44d1b13b479eb65e8c89 P 734e844b983d4bc0a5ce44ce146295b8: Bootstrap starting.
I20260812 02:41:28.728559 31543 tablet_bootstrap.cc:654] T e25edded890e44d1b13b479eb65e8c89 P f6c631bbe8ff4808a248d32f53ebd11b: Neither blocks nor log segments found. Creating new log.
I20260812 02:41:28.729934 31543 log.cc:824] T e25edded890e44d1b13b479eb65e8c89 P f6c631bbe8ff4808a248d32f53ebd11b: Log is configured to *not* fsync() on all Append() calls
I20260812 02:41:28.731263 31544 tablet_bootstrap.cc:654] T e25edded890e44d1b13b479eb65e8c89 P dedc1d8791af46b08b7f56ba65052e15: Neither blocks nor log segments found. Creating new log.
I20260812 02:41:28.731464 31545 tablet_bootstrap.cc:654] T e25edded890e44d1b13b479eb65e8c89 P 734e844b983d4bc0a5ce44ce146295b8: Neither blocks nor log segments found. Creating new log.
I20260812 02:41:28.732345 31544 log.cc:824] T e25edded890e44d1b13b479eb65e8c89 P dedc1d8791af46b08b7f56ba65052e15: Log is configured to *not* fsync() on all Append() calls
I20260812 02:41:28.732662 31545 log.cc:824] T e25edded890e44d1b13b479eb65e8c89 P 734e844b983d4bc0a5ce44ce146295b8: Log is configured to *not* fsync() on all Append() calls
I20260812 02:41:28.733358 31543 tablet_bootstrap.cc:492] T e25edded890e44d1b13b479eb65e8c89 P f6c631bbe8ff4808a248d32f53ebd11b: No bootstrap required, opened a new log
I20260812 02:41:28.733651 31543 ts_tablet_manager.cc:1397] T e25edded890e44d1b13b479eb65e8c89 P f6c631bbe8ff4808a248d32f53ebd11b: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260812 02:41:28.735296 31545 tablet_bootstrap.cc:492] T e25edded890e44d1b13b479eb65e8c89 P 734e844b983d4bc0a5ce44ce146295b8: No bootstrap required, opened a new log
I20260812 02:41:28.735509 31544 tablet_bootstrap.cc:492] T e25edded890e44d1b13b479eb65e8c89 P dedc1d8791af46b08b7f56ba65052e15: No bootstrap required, opened a new log
I20260812 02:41:28.735632 31545 ts_tablet_manager.cc:1397] T e25edded890e44d1b13b479eb65e8c89 P 734e844b983d4bc0a5ce44ce146295b8: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.006s
I20260812 02:41:28.735792 31544 ts_tablet_manager.cc:1397] T e25edded890e44d1b13b479eb65e8c89 P dedc1d8791af46b08b7f56ba65052e15: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.003s
I20260812 02:41:28.743209 31545 raft_consensus.cc:348] T e25edded890e44d1b13b479eb65e8c89 P 734e844b983d4bc0a5ce44ce146295b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "734e844b983d4bc0a5ce44ce146295b8" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 32877 } } peers { permanent_uuid: "f6c631bbe8ff4808a248d32f53ebd11b" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33485 } } peers { permanent_uuid: "dedc1d8791af46b08b7f56ba65052e15" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 43495 } }
I20260812 02:41:28.743577 31545 raft_consensus.cc:374] T e25edded890e44d1b13b479eb65e8c89 P 734e844b983d4bc0a5ce44ce146295b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:41:28.743681 31545 raft_consensus.cc:729] T e25edded890e44d1b13b479eb65e8c89 P 734e844b983d4bc0a5ce44ce146295b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 734e844b983d4bc0a5ce44ce146295b8, State: Initialized, Role: FOLLOWER
I20260812 02:41:28.744316 31545 consensus_queue.cc:260] T e25edded890e44d1b13b479eb65e8c89 P 734e844b983d4bc0a5ce44ce146295b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "734e844b983d4bc0a5ce44ce146295b8" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 32877 } } peers { permanent_uuid: "f6c631bbe8ff4808a248d32f53ebd11b" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33485 } } peers { permanent_uuid: "dedc1d8791af46b08b7f56ba65052e15" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 43495 } }
I20260812 02:41:28.744681 31543 raft_consensus.cc:348] T e25edded890e44d1b13b479eb65e8c89 P f6c631bbe8ff4808a248d32f53ebd11b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "734e844b983d4bc0a5ce44ce146295b8" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 32877 } } peers { permanent_uuid: "f6c631bbe8ff4808a248d32f53ebd11b" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33485 } } peers { permanent_uuid: "dedc1d8791af46b08b7f56ba65052e15" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 43495 } }
I20260812 02:41:28.745113 31543 raft_consensus.cc:374] T e25edded890e44d1b13b479eb65e8c89 P f6c631bbe8ff4808a248d32f53ebd11b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:41:28.745213 31543 raft_consensus.cc:729] T e25edded890e44d1b13b479eb65e8c89 P f6c631bbe8ff4808a248d32f53ebd11b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6c631bbe8ff4808a248d32f53ebd11b, State: Initialized, Role: FOLLOWER
I20260812 02:41:28.745528 31369 heartbeater.cc:503] Master 127.7.132.190:46677 was elected leader, sending a full tablet report...
I20260812 02:41:28.745873 31543 consensus_queue.cc:260] T e25edded890e44d1b13b479eb65e8c89 P f6c631bbe8ff4808a248d32f53ebd11b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "734e844b983d4bc0a5ce44ce146295b8" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 32877 } } peers { permanent_uuid: "f6c631bbe8ff4808a248d32f53ebd11b" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33485 } } peers { permanent_uuid: "dedc1d8791af46b08b7f56ba65052e15" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 43495 } }
I20260812 02:41:28.746163 31545 ts_tablet_manager.cc:1428] T e25edded890e44d1b13b479eb65e8c89 P 734e844b983d4bc0a5ce44ce146295b8: Time spent starting tablet: real 0.010s user 0.009s sys 0.003s
I20260812 02:41:28.746337 31544 raft_consensus.cc:348] T e25edded890e44d1b13b479eb65e8c89 P dedc1d8791af46b08b7f56ba65052e15 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "734e844b983d4bc0a5ce44ce146295b8" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 32877 } } peers { permanent_uuid: "f6c631bbe8ff4808a248d32f53ebd11b" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33485 } } peers { permanent_uuid: "dedc1d8791af46b08b7f56ba65052e15" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 43495 } }
I20260812 02:41:28.746868 31544 raft_consensus.cc:374] T e25edded890e44d1b13b479eb65e8c89 P dedc1d8791af46b08b7f56ba65052e15 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:41:28.746979 31544 raft_consensus.cc:729] T e25edded890e44d1b13b479eb65e8c89 P dedc1d8791af46b08b7f56ba65052e15 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dedc1d8791af46b08b7f56ba65052e15, State: Initialized, Role: FOLLOWER
I20260812 02:41:28.747766 31544 consensus_queue.cc:260] T e25edded890e44d1b13b479eb65e8c89 P dedc1d8791af46b08b7f56ba65052e15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "734e844b983d4bc0a5ce44ce146295b8" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 32877 } } peers { permanent_uuid: "f6c631bbe8ff4808a248d32f53ebd11b" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33485 } } peers { permanent_uuid: "dedc1d8791af46b08b7f56ba65052e15" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 43495 } }
I20260812 02:41:28.748662 31521 heartbeater.cc:503] Master 127.7.132.190:46677 was elected leader, sending a full tablet report...
I20260812 02:41:28.749333 31543 ts_tablet_manager.cc:1428] T e25edded890e44d1b13b479eb65e8c89 P f6c631bbe8ff4808a248d32f53ebd11b: Time spent starting tablet: real 0.016s user 0.017s sys 0.000s
I20260812 02:41:28.750869 31544 ts_tablet_manager.cc:1428] T e25edded890e44d1b13b479eb65e8c89 P dedc1d8791af46b08b7f56ba65052e15: Time spent starting tablet: real 0.015s user 0.009s sys 0.004s
W20260812 02:41:28.783471 31208 tablet.cc:2369] T e25edded890e44d1b13b479eb65e8c89 P dedc1d8791af46b08b7f56ba65052e15: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 02:41:28.831794 31370 tablet.cc:2369] T e25edded890e44d1b13b479eb65e8c89 P 734e844b983d4bc0a5ce44ce146295b8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 02:41:28.869861 31522 tablet.cc:2369] T e25edded890e44d1b13b479eb65e8c89 P f6c631bbe8ff4808a248d32f53ebd11b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 02:41:28.927405 31550 raft_consensus.cc:482] T e25edded890e44d1b13b479eb65e8c89 P 734e844b983d4bc0a5ce44ce146295b8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 02:41:28.927767 31550 raft_consensus.cc:504] T e25edded890e44d1b13b479eb65e8c89 P 734e844b983d4bc0a5ce44ce146295b8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "734e844b983d4bc0a5ce44ce146295b8" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 32877 } } peers { permanent_uuid: "f6c631bbe8ff4808a248d32f53ebd11b" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33485 } } peers { permanent_uuid: "dedc1d8791af46b08b7f56ba65052e15" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 43495 } }
I20260812 02:41:28.929551 31550 leader_election.cc:288] T e25edded890e44d1b13b479eb65e8c89 P 734e844b983d4bc0a5ce44ce146295b8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f6c631bbe8ff4808a248d32f53ebd11b (127.7.132.131:33485), dedc1d8791af46b08b7f56ba65052e15 (127.7.132.129:43495)
I20260812 02:41:28.937477 31541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:28.930253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:54441 (local address 127.7.132.129:43495)
0812 02:41:28.930498 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0812 02:41:28.930506 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:41:28.931128 (+ 622us) server_negotiation.cc:407] Connection header received
0812 02:41:28.931293 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:41:28.931298 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:41:28.931379 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:41:28.931477 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:41:28.932235 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:28.933031 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:28.935502 (+ 2471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:28.936400 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:28.936535 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:28.936589 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 02:41:28.937057 (+ 468us) server_negotiation.cc:299] Negotiation successful
0812 02:41:28.937203 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260812 02:41:28.937520 31540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:28.929934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:35359 (local address 127.7.132.131:33485)
0812 02:41:28.930181 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0812 02:41:28.930189 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:41:28.931281 (+ 1092us) server_negotiation.cc:407] Connection header received
0812 02:41:28.931497 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:41:28.931503 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:41:28.931580 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:41:28.931702 (+ 122us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:41:28.932413 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:28.933181 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:28.935621 (+ 2440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:28.936535 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:28.936723 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:28.936799 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 02:41:28.937140 (+ 341us) server_negotiation.cc:299] Negotiation successful
0812 02:41:28.937286 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 02:41:28.937595 31553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:28.930274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:43495 (local address 127.7.132.130:54441)
0812 02:41:28.931003 (+ 729us) negotiation.cc:107] Waiting for socket to connect
0812 02:41:28.931022 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 02:41:28.931148 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:41:28.931511 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:41:28.931516 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:41:28.931540 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:41:28.932059 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:41:28.932070 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:41:28.933181 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:41:28.933186 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:41:28.935342 (+ 2156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:41:28.935354 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:41:28.936666 (+ 1312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:41:28.936674 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:41:28.936855 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:28.936910 (+ 55us) client_negotiation.cc:769] Sending connection context
0812 02:41:28.936988 (+ 78us) client_negotiation.cc:240] Negotiation successful
0812 02:41:28.937131 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":620,"mutex_wait_us":55,"thread_start_us":373,"threads_started":1}
I20260812 02:41:28.937844 31554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:28.930287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.131:33485 (local address 127.7.132.130:35359)
0812 02:41:28.931161 (+ 874us) negotiation.cc:107] Waiting for socket to connect
0812 02:41:28.931179 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 02:41:28.931338 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:41:28.931733 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:41:28.931736 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:41:28.931756 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:41:28.932252 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:41:28.932264 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:41:28.933332 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:41:28.933336 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:41:28.935403 (+ 2067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:41:28.935438 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:41:28.936757 (+ 1319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:41:28.936761 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:41:28.936942 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:28.936988 (+ 46us) client_negotiation.cc:769] Sending connection context
0812 02:41:28.937091 (+ 103us) client_negotiation.cc:240] Negotiation successful
0812 02:41:28.937214 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":730,"mutex_wait_us":77,"thread_start_us":475,"threads_started":1}
I20260812 02:41:28.938691 31152 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e25edded890e44d1b13b479eb65e8c89" candidate_uuid: "734e844b983d4bc0a5ce44ce146295b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dedc1d8791af46b08b7f56ba65052e15" is_pre_election: true
I20260812 02:41:28.938864 31471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e25edded890e44d1b13b479eb65e8c89" candidate_uuid: "734e844b983d4bc0a5ce44ce146295b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6c631bbe8ff4808a248d32f53ebd11b" is_pre_election: true
I20260812 02:41:28.939265 31152 raft_consensus.cc:2454] T e25edded890e44d1b13b479eb65e8c89 P dedc1d8791af46b08b7f56ba65052e15 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 734e844b983d4bc0a5ce44ce146295b8 in term 0.
I20260812 02:41:28.939476 31471 raft_consensus.cc:2454] T e25edded890e44d1b13b479eb65e8c89 P f6c631bbe8ff4808a248d32f53ebd11b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 734e844b983d4bc0a5ce44ce146295b8 in term 0.
I20260812 02:41:28.939998 31234 leader_election.cc:302] T e25edded890e44d1b13b479eb65e8c89 P 734e844b983d4bc0a5ce44ce146295b8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 734e844b983d4bc0a5ce44ce146295b8, dedc1d8791af46b08b7f56ba65052e15; no voters:
I20260812 02:41:28.940420 31550 raft_consensus.cc:2781] T e25edded890e44d1b13b479eb65e8c89 P 734e844b983d4bc0a5ce44ce146295b8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 02:41:28.940579 31550 raft_consensus.cc:482] T e25edded890e44d1b13b479eb65e8c89 P 734e844b983d4bc0a5ce44ce146295b8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 02:41:28.940682 31550 raft_consensus.cc:3037] T e25edded890e44d1b13b479eb65e8c89 P 734e844b983d4bc0a5ce44ce146295b8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:41:28.942981 31550 raft_consensus.cc:504] T e25edded890e44d1b13b479eb65e8c89 P 734e844b983d4bc0a5ce44ce146295b8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "734e844b983d4bc0a5ce44ce146295b8" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 32877 } } peers { permanent_uuid: "f6c631bbe8ff4808a248d32f53ebd11b" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33485 } } peers { permanent_uuid: "dedc1d8791af46b08b7f56ba65052e15" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 43495 } }
I20260812 02:41:28.943646 31550 leader_election.cc:288] T e25edded890e44d1b13b479eb65e8c89 P 734e844b983d4bc0a5ce44ce146295b8 [CANDIDATE]: Term 1 election: Requested vote from peers f6c631bbe8ff4808a248d32f53ebd11b (127.7.132.131:33485), dedc1d8791af46b08b7f56ba65052e15 (127.7.132.129:43495)
I20260812 02:41:28.944101 31471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e25edded890e44d1b13b479eb65e8c89" candidate_uuid: "734e844b983d4bc0a5ce44ce146295b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6c631bbe8ff4808a248d32f53ebd11b"
I20260812 02:41:28.944187 31152 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e25edded890e44d1b13b479eb65e8c89" candidate_uuid: "734e844b983d4bc0a5ce44ce146295b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dedc1d8791af46b08b7f56ba65052e15"
I20260812 02:41:28.944326 31471 raft_consensus.cc:3037] T e25edded890e44d1b13b479eb65e8c89 P f6c631bbe8ff4808a248d32f53ebd11b [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:41:28.944434 31152 raft_consensus.cc:3037] T e25edded890e44d1b13b479eb65e8c89 P dedc1d8791af46b08b7f56ba65052e15 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:41:28.946713 31471 raft_consensus.cc:2454] T e25edded890e44d1b13b479eb65e8c89 P f6c631bbe8ff4808a248d32f53ebd11b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 734e844b983d4bc0a5ce44ce146295b8 in term 1.
I20260812 02:41:28.946799 31152 raft_consensus.cc:2454] T e25edded890e44d1b13b479eb65e8c89 P dedc1d8791af46b08b7f56ba65052e15 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 734e844b983d4bc0a5ce44ce146295b8 in term 1.
I20260812 02:41:28.947235 31231 leader_election.cc:302] T e25edded890e44d1b13b479eb65e8c89 P 734e844b983d4bc0a5ce44ce146295b8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 734e844b983d4bc0a5ce44ce146295b8, f6c631bbe8ff4808a248d32f53ebd11b; no voters:
I20260812 02:41:28.947584 31550 raft_consensus.cc:2781] T e25edded890e44d1b13b479eb65e8c89 P 734e844b983d4bc0a5ce44ce146295b8 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:41:28.948063 31550 raft_consensus.cc:686] T e25edded890e44d1b13b479eb65e8c89 P 734e844b983d4bc0a5ce44ce146295b8 [term 1 LEADER]: Becoming Leader. State: Replica: 734e844b983d4bc0a5ce44ce146295b8, State: Running, Role: LEADER
I20260812 02:41:28.948632 31550 consensus_queue.cc:237] T e25edded890e44d1b13b479eb65e8c89 P 734e844b983d4bc0a5ce44ce146295b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "734e844b983d4bc0a5ce44ce146295b8" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 32877 } } peers { permanent_uuid: "f6c631bbe8ff4808a248d32f53ebd11b" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33485 } } peers { permanent_uuid: "dedc1d8791af46b08b7f56ba65052e15" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 43495 } }
I20260812 02:41:28.953393 30983 catalog_manager.cc:5495] T e25edded890e44d1b13b479eb65e8c89 P 734e844b983d4bc0a5ce44ce146295b8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 734e844b983d4bc0a5ce44ce146295b8 (127.7.132.130). New cstate: current_term: 1 leader_uuid: "734e844b983d4bc0a5ce44ce146295b8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "734e844b983d4bc0a5ce44ce146295b8" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 32877 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f6c631bbe8ff4808a248d32f53ebd11b" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33485 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dedc1d8791af46b08b7f56ba65052e15" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 43495 } health_report { overall_health: UNKNOWN } } }
I20260812 02:41:29.004599 31542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:28.999292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40686 (local address 127.7.132.130:32877)
0812 02:41:28.999529 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0812 02:41:28.999534 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:41:28.999552 (+ 18us) server_negotiation.cc:407] Connection header received
0812 02:41:28.999718 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:41:28.999722 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:41:28.999814 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:41:28.999942 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 02:41:29.000317 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:29.001032 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:29.002569 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:29.002960 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:29.003058 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:29.003486 (+ 428us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 02:41:29.004006 (+ 520us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 02:41:29.004129 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0812 02:41:29.004207 (+ 78us) server_negotiation.cc:299] Negotiation successful
0812 02:41:29.004277 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260812 02:41:29.012909 31471 raft_consensus.cc:1261] T e25edded890e44d1b13b479eb65e8c89 P f6c631bbe8ff4808a248d32f53ebd11b [term 1 FOLLOWER]: Refusing update from remote peer 734e844b983d4bc0a5ce44ce146295b8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:41:29.012910 31152 raft_consensus.cc:1261] T e25edded890e44d1b13b479eb65e8c89 P dedc1d8791af46b08b7f56ba65052e15 [term 1 FOLLOWER]: Refusing update from remote peer 734e844b983d4bc0a5ce44ce146295b8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:41:29.013942 31555 consensus_queue.cc:1035] T e25edded890e44d1b13b479eb65e8c89 P 734e844b983d4bc0a5ce44ce146295b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6c631bbe8ff4808a248d32f53ebd11b" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33485 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:41:29.014325 31550 consensus_queue.cc:1035] T e25edded890e44d1b13b479eb65e8c89 P 734e844b983d4bc0a5ce44ce146295b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dedc1d8791af46b08b7f56ba65052e15" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 43495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:41:29.026100 31560 mvcc.cc:205] Tried to move back new op lower bound from 7317514194986872832 to 7317514194737848320. Current Snapshot: MvccSnapshot[applied={T|T < 7317514194986872832}]
I20260812 02:41:29.033782 31561 mvcc.cc:205] Tried to move back new op lower bound from 7317514194986872832 to 7317514194737848320. Current Snapshot: MvccSnapshot[applied={T|T < 7317514194986872832}]
I20260812 02:41:29.045281 30984 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56256:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 02:41:29.045624 30984 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56256:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 02:41:29.049161 30984 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9a9539077c254ad48d0c621da3924a34: Sending DeleteTablet for 3 replicas of tablet e25edded890e44d1b13b479eb65e8c89
I20260812 02:41:29.050315 31292 tablet_service.cc:1514] Processing DeleteTablet for tablet e25edded890e44d1b13b479eb65e8c89 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:41:29 UTC) from {username='kudu', principal='kudu/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:40678
I20260812 02:41:29.050446 31448 tablet_service.cc:1514] Processing DeleteTablet for tablet e25edded890e44d1b13b479eb65e8c89 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:41:29 UTC) from {username='kudu', principal='kudu/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:41574
I20260812 02:41:29.050505 31127 tablet_service.cc:1514] Processing DeleteTablet for tablet e25edded890e44d1b13b479eb65e8c89 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:41:29 UTC) from {username='kudu', principal='kudu/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:43726
I20260812 02:41:29.053849 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 31055
I20260812 02:41:29.061784 31574 tablet_replica.cc:330] T e25edded890e44d1b13b479eb65e8c89 P dedc1d8791af46b08b7f56ba65052e15: stopping tablet replica
I20260812 02:41:29.062810 31574 raft_consensus.cc:2229] T e25edded890e44d1b13b479eb65e8c89 P dedc1d8791af46b08b7f56ba65052e15 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 02:41:29.063546 31574 raft_consensus.cc:2258] T e25edded890e44d1b13b479eb65e8c89 P dedc1d8791af46b08b7f56ba65052e15 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:41:29.064302 31573 tablet_replica.cc:330] T e25edded890e44d1b13b479eb65e8c89 P f6c631bbe8ff4808a248d32f53ebd11b: stopping tablet replica
I20260812 02:41:29.064741 31573 raft_consensus.cc:2229] T e25edded890e44d1b13b479eb65e8c89 P f6c631bbe8ff4808a248d32f53ebd11b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 02:41:29.065251 31573 raft_consensus.cc:2258] T e25edded890e44d1b13b479eb65e8c89 P f6c631bbe8ff4808a248d32f53ebd11b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:41:29.066051 31574 ts_tablet_manager.cc:1905] T e25edded890e44d1b13b479eb65e8c89 P dedc1d8791af46b08b7f56ba65052e15: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:41:29.066514 31571 tablet_replica.cc:330] T e25edded890e44d1b13b479eb65e8c89 P 734e844b983d4bc0a5ce44ce146295b8: stopping tablet replica
I20260812 02:41:29.066936 31571 raft_consensus.cc:2229] T e25edded890e44d1b13b479eb65e8c89 P 734e844b983d4bc0a5ce44ce146295b8 [term 1 LEADER]: Raft consensus shutting down.
I20260812 02:41:29.067596 31571 raft_consensus.cc:2258] T e25edded890e44d1b13b479eb65e8c89 P 734e844b983d4bc0a5ce44ce146295b8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:41:29.069792 31571 ts_tablet_manager.cc:1905] T e25edded890e44d1b13b479eb65e8c89 P 734e844b983d4bc0a5ce44ce146295b8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:41:29.070621 31573 ts_tablet_manager.cc:1905] T e25edded890e44d1b13b479eb65e8c89 P f6c631bbe8ff4808a248d32f53ebd11b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:41:29.073329 31574 ts_tablet_manager.cc:1918] T e25edded890e44d1b13b479eb65e8c89 P dedc1d8791af46b08b7f56ba65052e15: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 02:41:29.073516 31574 log.cc:1196] T e25edded890e44d1b13b479eb65e8c89 P dedc1d8791af46b08b7f56ba65052e15: Deleting WAL directory at /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502156971023-7698-0/minicluster-data/ts-0/wal/wals/e25edded890e44d1b13b479eb65e8c89
I20260812 02:41:29.074204 31574 ts_tablet_manager.cc:1939] T e25edded890e44d1b13b479eb65e8c89 P dedc1d8791af46b08b7f56ba65052e15: Deleting consensus metadata
I20260812 02:41:29.076598 30966 catalog_manager.cc:4841] TS dedc1d8791af46b08b7f56ba65052e15 (127.7.132.129:43495): tablet e25edded890e44d1b13b479eb65e8c89 (table test-table [id=e8fc0b59ff414c4b9221c94774f9e3d1]) successfully deleted
I20260812 02:41:29.077375 31571 ts_tablet_manager.cc:1918] T e25edded890e44d1b13b479eb65e8c89 P 734e844b983d4bc0a5ce44ce146295b8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 02:41:29.077708 31571 log.cc:1196] T e25edded890e44d1b13b479eb65e8c89 P 734e844b983d4bc0a5ce44ce146295b8: Deleting WAL directory at /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502156971023-7698-0/minicluster-data/ts-1/wal/wals/e25edded890e44d1b13b479eb65e8c89
I20260812 02:41:29.078321 31571 ts_tablet_manager.cc:1939] T e25edded890e44d1b13b479eb65e8c89 P 734e844b983d4bc0a5ce44ce146295b8: Deleting consensus metadata
Aug 12 02:41:29 dist-test-slave-mn4r krb5kdc[30930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502486, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.132.129@KRBTEST.COM
I20260812 02:41:29.080235 30965 catalog_manager.cc:4841] TS 734e844b983d4bc0a5ce44ce146295b8 (127.7.132.130:32877): tablet e25edded890e44d1b13b479eb65e8c89 (table test-table [id=e8fc0b59ff414c4b9221c94774f9e3d1]) successfully deleted
I20260812 02:41:29.084040 31573 ts_tablet_manager.cc:1918] T e25edded890e44d1b13b479eb65e8c89 P f6c631bbe8ff4808a248d32f53ebd11b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 02:41:29.084282 31573 log.cc:1196] T e25edded890e44d1b13b479eb65e8c89 P f6c631bbe8ff4808a248d32f53ebd11b: Deleting WAL directory at /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786502156971023-7698-0/minicluster-data/ts-2/wal/wals/e25edded890e44d1b13b479eb65e8c89
I20260812 02:41:29.084867 31573 ts_tablet_manager.cc:1939] T e25edded890e44d1b13b479eb65e8c89 P f6c631bbe8ff4808a248d32f53ebd11b: Deleting consensus metadata
I20260812 02:41:29.086858 30963 catalog_manager.cc:4841] TS f6c631bbe8ff4808a248d32f53ebd11b (127.7.132.131:33485): tablet e25edded890e44d1b13b479eb65e8c89 (table test-table [id=e8fc0b59ff414c4b9221c94774f9e3d1]) successfully deleted
I20260812 02:41:29.089448 31541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:29.056543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43728 (local address 127.7.132.129:43495)
0812 02:41:29.056730 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0812 02:41:29.056738 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:41:29.061593 (+ 4855us) server_negotiation.cc:407] Connection header received
0812 02:41:29.061802 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:41:29.061807 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:41:29.061927 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:41:29.062049 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:41:29.065482 (+ 3433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:29.066682 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:29.073601 (+ 6919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:29.074353 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:29.074464 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:29.081709 (+ 7245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:41:29.081728 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:41:29.081748 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:41:29.081794 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:41:29.085789 (+ 3995us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:41:29.086720 (+ 931us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:41:29.086731 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:41:29.086734 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:41:29.086911 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:41:29.087388 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:41:29.087395 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:41:29.087399 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:41:29.087877 (+ 478us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:41:29.088205 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0812 02:41:29.088578 (+ 373us) server_negotiation.cc:299] Negotiation successful
0812 02:41:29.088834 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260812 02:41:29.090202 31201 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:41:29.244831 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 31055
I20260812 02:41:29.270344 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 31213
Aug 12 02:41:29 dist-test-slave-mn4r krb5kdc[30930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502486, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.132.130@KRBTEST.COM
I20260812 02:41:29.294620 31542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:29.271283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40690 (local address 127.7.132.130:32877)
0812 02:41:29.271543 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0812 02:41:29.271551 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:41:29.271573 (+ 22us) server_negotiation.cc:407] Connection header received
0812 02:41:29.271731 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:41:29.271751 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:41:29.271888 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:41:29.272015 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:41:29.273763 (+ 1748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:29.274692 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:29.276591 (+ 1899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:29.277042 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:29.277141 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:29.288028 (+ 10887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:41:29.288043 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:41:29.288060 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:41:29.288107 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:41:29.291355 (+ 3248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:41:29.292230 (+ 875us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:41:29.292238 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:41:29.292240 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:41:29.292386 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:41:29.292819 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:41:29.292823 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:41:29.292825 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:41:29.293193 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:41:29.293457 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0812 02:41:29.293792 (+ 335us) server_negotiation.cc:299] Negotiation successful
0812 02:41:29.293965 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114}
I20260812 02:41:29.295472 31364 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:41:29.422240 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 31213
I20260812 02:41:29.446412 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 31375
Aug 12 02:41:29 dist-test-slave-mn4r krb5kdc[30930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502486, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.132.131@KRBTEST.COM
I20260812 02:41:29.465693 31578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:29.447206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41590 (local address 127.7.132.131:33485)
0812 02:41:29.447749 (+ 543us) server_negotiation.cc:206] Beginning negotiation
0812 02:41:29.447753 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 02:41:29.447770 (+ 17us) server_negotiation.cc:407] Connection header received
0812 02:41:29.447828 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:41:29.447832 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:41:29.447893 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:41:29.447985 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:41:29.449355 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:29.450065 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:29.451368 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:29.451750 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:29.451841 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:29.457367 (+ 5526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:41:29.457390 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:41:29.457414 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:41:29.457479 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:41:29.462054 (+ 4575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:41:29.463164 (+ 1110us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:41:29.463176 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:41:29.463182 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:41:29.463361 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:41:29.463813 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:41:29.463821 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:41:29.463828 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:41:29.464265 (+ 437us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:41:29.464486 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0812 02:41:29.465018 (+ 532us) server_negotiation.cc:299] Negotiation successful
0812 02:41:29.465292 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":443,"thread_start_us":340,"threads_started":1}
I20260812 02:41:29.466686 31516 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:41:29.568145 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 31375
I20260812 02:41:29.588891 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 30946
I20260812 02:41:29.590058 31021 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:41:29.701485 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 30946
2026-08-12T02:41:29Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (4885 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-12T02:41:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T02:41:29Z Disabled control of system clock
I20260812 02:41:29.797935 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502156971023-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502156971023-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502156971023-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:40105
--webserver_interface=127.7.132.190
--webserver_port=0
--builtin_ntp_servers=127.7.132.148:32897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.132.190:40105
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 02:41:30.152626 31586 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:41:30.152922 31586 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:41:30.153007 31586 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:41:30.160292 31586 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:41:30.160393 31586 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:41:30.160436 31586 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:41:30.160499 31586 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:41:30.160549 31586 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:41:30.169775 31586 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:32897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502156971023-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502156971023-7698-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.132.190:40105
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:40105
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.132.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502156971023-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:41:30.171103 31586 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:41:30.172879 31586 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:41:30.184852 31595 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:41:30.184980 31592 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:41:30.184896 31593 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:41:30.186201 31586 server_base.cc:1034] running on GCE node
I20260812 02:41:30.186995 31586 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:41:30.188405 31586 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:41:30.189654 31586 hybrid_clock.cc:648] HybridClock initialized: now 1786502490189625 us; error 48 us; skew 500 ppm
I20260812 02:41:30.190023 31586 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 02:41:30.192333 31586 webserver.cc:460] Webserver started at http://127.7.132.190:35737/ using document root <none> and password file <none>
I20260812 02:41:30.193038 31586 fs_manager.cc:359] Metadata directory not provided
I20260812 02:41:30.193187 31586 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:41:30.193524 31586 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:41:30.196578 31586 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502156971023-7698-0/minicluster-data/master-0/data/instance:
uuid: "e768c0fc1ae946cb879ef98dbf3d6e30"
format_stamp: "Formatted at 2026-08-12 02:41:30 on dist-test-slave-mn4r"
server_key: "c731ca002878f30d6af56f7d725d4908"
server_key_iv: "c6ef354a041dcfec64175aa5f2b36e68"
server_key_version: "encryptionkey@0"
I20260812 02:41:30.197443 31586 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502156971023-7698-0/minicluster-data/master-0/wal/instance:
uuid: "e768c0fc1ae946cb879ef98dbf3d6e30"
format_stamp: "Formatted at 2026-08-12 02:41:30 on dist-test-slave-mn4r"
server_key: "c731ca002878f30d6af56f7d725d4908"
server_key_iv: "c6ef354a041dcfec64175aa5f2b36e68"
server_key_version: "encryptionkey@0"
I20260812 02:41:30.202986 31586 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.006s
I20260812 02:41:30.206688 31602 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:41:30.208137 31586 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 02:41:30.208344 31586 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502156971023-7698-0/minicluster-data/master-0/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502156971023-7698-0/minicluster-data/master-0/wal
uuid: "e768c0fc1ae946cb879ef98dbf3d6e30"
format_stamp: "Formatted at 2026-08-12 02:41:30 on dist-test-slave-mn4r"
server_key: "c731ca002878f30d6af56f7d725d4908"
server_key_iv: "c6ef354a041dcfec64175aa5f2b36e68"
server_key_version: "encryptionkey@0"
I20260812 02:41:30.208577 31586 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502156971023-7698-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502156971023-7698-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502156971023-7698-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:41:30.221747 31586 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:41:30.222702 31586 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:41:30.223006 31586 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:41:30.242093 31586 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.190:40105
I20260812 02:41:30.242192 31664 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.190:40105 every 8 connection(s)
I20260812 02:41:30.243475 31586 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
I20260812 02:41:30.247193 31665 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:41:30.248551 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 31586
I20260812 02:41:30.248745 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502156971023-7698-0/minicluster-data/master-0/wal/instance
I20260812 02:41:30.249238 7698 external_mini_cluster.cc:1428] Setting key ed1be02a0252d92740df455758776322
I20260812 02:41:30.261121 31665 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e768c0fc1ae946cb879ef98dbf3d6e30: Bootstrap starting.
I20260812 02:41:30.263058 31669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:30.251188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60648 (local address 127.7.132.190:40105)
0812 02:41:30.252288 (+ 1100us) server_negotiation.cc:206] Beginning negotiation
0812 02:41:30.252308 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0812 02:41:30.252358 (+ 50us) server_negotiation.cc:407] Connection header received
0812 02:41:30.253897 (+ 1539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:41:30.253923 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:41:30.254468 (+ 545us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:41:30.254797 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:41:30.255369 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:30.256655 (+ 1286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:30.258587 (+ 1932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:30.259267 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:30.259395 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:30.260649 (+ 1254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:41:30.260656 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:41:30.260669 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:41:30.261214 (+ 545us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:41:30.261343 (+ 129us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:41:30.261367 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:41:30.261461 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0812 02:41:30.261705 (+ 244us) server_negotiation.cc:299] Negotiation successful
0812 02:41:30.262050 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":693,"thread_start_us":390,"threads_started":1}
I20260812 02:41:30.264606 31665 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e768c0fc1ae946cb879ef98dbf3d6e30: Neither blocks nor log segments found. Creating new log.
I20260812 02:41:30.265818 31665 log.cc:824] T 00000000000000000000000000000000 P e768c0fc1ae946cb879ef98dbf3d6e30: Log is configured to *not* fsync() on all Append() calls
I20260812 02:41:30.268515 31665 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e768c0fc1ae946cb879ef98dbf3d6e30: No bootstrap required, opened a new log
I20260812 02:41:30.275767 31665 raft_consensus.cc:348] T 00000000000000000000000000000000 P e768c0fc1ae946cb879ef98dbf3d6e30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e768c0fc1ae946cb879ef98dbf3d6e30" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 40105 } }
I20260812 02:41:30.276154 31665 raft_consensus.cc:374] T 00000000000000000000000000000000 P e768c0fc1ae946cb879ef98dbf3d6e30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:41:30.276265 31665 raft_consensus.cc:729] T 00000000000000000000000000000000 P e768c0fc1ae946cb879ef98dbf3d6e30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e768c0fc1ae946cb879ef98dbf3d6e30, State: Initialized, Role: FOLLOWER
I20260812 02:41:30.276822 31665 consensus_queue.cc:260] T 00000000000000000000000000000000 P e768c0fc1ae946cb879ef98dbf3d6e30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e768c0fc1ae946cb879ef98dbf3d6e30" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 40105 } }
I20260812 02:41:30.277077 31665 raft_consensus.cc:388] T 00000000000000000000000000000000 P e768c0fc1ae946cb879ef98dbf3d6e30 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 02:41:30.277225 31665 raft_consensus.cc:482] T 00000000000000000000000000000000 P e768c0fc1ae946cb879ef98dbf3d6e30 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 02:41:30.277403 31665 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e768c0fc1ae946cb879ef98dbf3d6e30 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:41:30.279902 31665 raft_consensus.cc:504] T 00000000000000000000000000000000 P e768c0fc1ae946cb879ef98dbf3d6e30 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e768c0fc1ae946cb879ef98dbf3d6e30" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 40105 } }
I20260812 02:41:30.280422 31665 leader_election.cc:302] T 00000000000000000000000000000000 P e768c0fc1ae946cb879ef98dbf3d6e30 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e768c0fc1ae946cb879ef98dbf3d6e30; no voters:
I20260812 02:41:30.281018 31665 leader_election.cc:288] T 00000000000000000000000000000000 P e768c0fc1ae946cb879ef98dbf3d6e30 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 02:41:30.281181 31671 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e768c0fc1ae946cb879ef98dbf3d6e30 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:41:30.282461 31671 raft_consensus.cc:686] T 00000000000000000000000000000000 P e768c0fc1ae946cb879ef98dbf3d6e30 [term 1 LEADER]: Becoming Leader. State: Replica: e768c0fc1ae946cb879ef98dbf3d6e30, State: Running, Role: LEADER
I20260812 02:41:30.283020 31671 consensus_queue.cc:237] T 00000000000000000000000000000000 P e768c0fc1ae946cb879ef98dbf3d6e30 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e768c0fc1ae946cb879ef98dbf3d6e30" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 40105 } }
I20260812 02:41:30.283324 31665 sys_catalog.cc:564] T 00000000000000000000000000000000 P e768c0fc1ae946cb879ef98dbf3d6e30 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:41:30.285789 31674 sys_catalog.cc:455] T 00000000000000000000000000000000 P e768c0fc1ae946cb879ef98dbf3d6e30 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e768c0fc1ae946cb879ef98dbf3d6e30. Latest consensus state: current_term: 1 leader_uuid: "e768c0fc1ae946cb879ef98dbf3d6e30" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e768c0fc1ae946cb879ef98dbf3d6e30" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 40105 } } }
I20260812 02:41:30.286296 31674 sys_catalog.cc:458] T 00000000000000000000000000000000 P e768c0fc1ae946cb879ef98dbf3d6e30 [sys.catalog]: This master's current role is: LEADER
I20260812 02:41:30.286268 31672 sys_catalog.cc:455] T 00000000000000000000000000000000 P e768c0fc1ae946cb879ef98dbf3d6e30 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e768c0fc1ae946cb879ef98dbf3d6e30" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e768c0fc1ae946cb879ef98dbf3d6e30" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 40105 } } }
I20260812 02:41:30.286566 31672 sys_catalog.cc:458] T 00000000000000000000000000000000 P e768c0fc1ae946cb879ef98dbf3d6e30 [sys.catalog]: This master's current role is: LEADER
I20260812 02:41:30.290994 31681 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:41:30.296387 31681 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 02:41:30.305202 31681 catalog_manager.cc:1347] Generated new cluster ID: 220e696ac85f4729877eb5b19d69fa22
I20260812 02:41:30.305322 31681 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 02:41:30.342365 31681 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 02:41:30.343734 31681 catalog_manager.cc:1504] Loading token signing keys...
I20260812 02:41:30.352000 31681 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e768c0fc1ae946cb879ef98dbf3d6e30: Generated new TSK 0
I20260812 02:41:30.352864 31681 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 02:41:30.361680 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502156971023-7698-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502156971023-7698-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502156971023-7698-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502156971023-7698-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.129:0
--local_ip_for_outbound_sockets=127.7.132.129
--webserver_interface=127.7.132.129
--webserver_port=0
--tserver_master_addrs=127.7.132.190:40105
--builtin_ntp_servers=127.7.132.148:32897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 02:41:30.720928 31697 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:41:30.721222 31697 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:41:30.721390 31697 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:41:30.728893 31697 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:41:30.728991 31697 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:41:30.729204 31697 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.129
I20260812 02:41:30.738333 31697 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:32897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502156971023-7698-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502156971023-7698-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502156971023-7698-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.132.129
--webserver_port=0
--tserver_master_addrs=127.7.132.190:40105
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.129
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502156971023-7698-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:41:30.739639 31697 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:41:30.741529 31697 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:41:30.755096 31703 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:41:30.755075 31702 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:41:30.756009 31697 server_base.cc:1034] running on GCE node
W20260812 02:41:30.756265 31705 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:41:30.756942 31697 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:41:30.757934 31697 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:41:30.759183 31697 hybrid_clock.cc:648] HybridClock initialized: now 1786502490759123 us; error 46 us; skew 500 ppm
I20260812 02:41:30.759558 31697 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 02:41:30.761871 31697 webserver.cc:460] Webserver started at http://127.7.132.129:41359/ using document root <none> and password file <none>
I20260812 02:41:30.762693 31697 fs_manager.cc:359] Metadata directory not provided
I20260812 02:41:30.762833 31697 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:41:30.763186 31697 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:41:30.766278 31697 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502156971023-7698-0/minicluster-data/ts-0/data/instance:
uuid: "218fa25fae394acda26b0d08283fd9dd"
format_stamp: "Formatted at 2026-08-12 02:41:30 on dist-test-slave-mn4r"
server_key: "401be6052210791e72f8001e1a00f9b7"
server_key_iv: "cff1cce2b8aa23eb13334e1cd13aa7b1"
server_key_version: "encryptionkey@0"
I20260812 02:41:30.767076 31697 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502156971023-7698-0/minicluster-data/ts-0/wal/instance:
uuid: "218fa25fae394acda26b0d08283fd9dd"
format_stamp: "Formatted at 2026-08-12 02:41:30 on dist-test-slave-mn4r"
server_key: "401be6052210791e72f8001e1a00f9b7"
server_key_iv: "cff1cce2b8aa23eb13334e1cd13aa7b1"
server_key_version: "encryptionkey@0"
I20260812 02:41:30.772207 31697 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 02:41:30.775877 31712 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:41:30.777294 31697 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 02:41:30.777482 31697 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502156971023-7698-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502156971023-7698-0/minicluster-data/ts-0/wal
uuid: "218fa25fae394acda26b0d08283fd9dd"
format_stamp: "Formatted at 2026-08-12 02:41:30 on dist-test-slave-mn4r"
server_key: "401be6052210791e72f8001e1a00f9b7"
server_key_iv: "cff1cce2b8aa23eb13334e1cd13aa7b1"
server_key_version: "encryptionkey@0"
I20260812 02:41:30.777727 31697 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502156971023-7698-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502156971023-7698-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502156971023-7698-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:41:30.789103 31697 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:41:30.789903 31697 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:41:30.790277 31697 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:41:30.791242 31697 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:41:30.792719 31697 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:41:30.792856 31697 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:41:30.792990 31697 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:41:30.793116 31697 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:41:30.827121 31697 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.129:43127
I20260812 02:41:30.827175 31842 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.129:43127 every 8 connection(s)
I20260812 02:41:30.828583 31697 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502156971023-7698-0/minicluster-data/ts-0/data/info.pb
I20260812 02:41:30.834539 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 31697
I20260812 02:41:30.834748 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502156971023-7698-0/minicluster-data/ts-0/wal/instance
I20260812 02:41:30.835350 7698 external_mini_cluster.cc:1428] Setting key 6a31cc2f083a533458d22a34302ad39d
I20260812 02:41:30.838438 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502156971023-7698-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502156971023-7698-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502156971023-7698-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502156971023-7698-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.130:0
--local_ip_for_outbound_sockets=127.7.132.130
--webserver_interface=127.7.132.130
--webserver_port=0
--tserver_master_addrs=127.7.132.190:40105
--builtin_ntp_servers=127.7.132.148:32897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 02:41:30.842479 31845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:30.831265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:56095 (local address 127.7.132.190:40105)
0812 02:41:30.831743 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0812 02:41:30.831749 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:41:30.832934 (+ 1185us) server_negotiation.cc:407] Connection header received
0812 02:41:30.835007 (+ 2073us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:41:30.835011 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:41:30.835102 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:41:30.835199 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:41:30.836542 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:30.837521 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:30.839265 (+ 1744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:30.840019 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:30.840182 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:30.841480 (+ 1298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:41:30.841489 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:41:30.841497 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:41:30.841522 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:41:30.841576 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:41:30.841595 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:41:30.841692 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0812 02:41:30.841908 (+ 216us) server_negotiation.cc:299] Negotiation successful
0812 02:41:30.842020 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":227,"threads_started":1}
I20260812 02:41:30.843359 31847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:30.831907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:40105 (local address 127.7.132.129:56095)
0812 02:41:30.832777 (+ 870us) negotiation.cc:107] Waiting for socket to connect
0812 02:41:30.832820 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 02:41:30.834707 (+ 1887us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:41:30.835473 (+ 766us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:41:30.835486 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:41:30.835608 (+ 122us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:41:30.836297 (+ 689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:41:30.836315 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:41:30.837675 (+ 1360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:41:30.837679 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:41:30.839119 (+ 1440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:41:30.839131 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:41:30.840234 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:41:30.840239 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:41:30.840437 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:30.841155 (+ 718us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 02:41:30.841157 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:41:30.841197 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 02:41:30.841276 (+ 79us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:41:30.841696 (+ 420us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:41:30.841705 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:41:30.841711 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 02:41:30.841851 (+ 140us) client_negotiation.cc:240] Negotiation successful
0812 02:41:30.842267 (+ 416us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":266,"threads_started":1}
I20260812 02:41:30.844918 31843 heartbeater.cc:347] Connected to a master server at 127.7.132.190:40105
I20260812 02:41:30.845263 31843 heartbeater.cc:464] Registering TS with master...
I20260812 02:41:30.846202 31843 heartbeater.cc:511] Master 127.7.132.190:40105 requested a full tablet report, sending...
I20260812 02:41:30.848420 31623 ts_manager.cc:194] Registered new tserver with Master: 218fa25fae394acda26b0d08283fd9dd (127.7.132.129:43127)
I20260812 02:41:30.849905 31623 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.132.129:56095
W20260812 02:41:31.182235 31849 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:41:31.182531 31849 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:41:31.182700 31849 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:41:31.190104 31849 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:41:31.190207 31849 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:41:31.190403 31849 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.130
I20260812 02:41:31.199491 31849 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:32897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502156971023-7698-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502156971023-7698-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502156971023-7698-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.132.130
--webserver_port=0
--tserver_master_addrs=127.7.132.190:40105
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.130
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502156971023-7698-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:41:31.200855 31849 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:41:31.202819 31849 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:41:31.216780 31855 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:41:31.216747 31856 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:41:31.216846 31859 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:41:31.217756 31849 server_base.cc:1034] running on GCE node
I20260812 02:41:31.218505 31849 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:41:31.219560 31849 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:41:31.220832 31849 hybrid_clock.cc:648] HybridClock initialized: now 1786502491220723 us; error 84 us; skew 500 ppm
I20260812 02:41:31.221230 31849 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 02:41:31.223773 31849 webserver.cc:460] Webserver started at http://127.7.132.130:45225/ using document root <none> and password file <none>
I20260812 02:41:31.224565 31849 fs_manager.cc:359] Metadata directory not provided
I20260812 02:41:31.224695 31849 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:41:31.225073 31849 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:41:31.228420 31849 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502156971023-7698-0/minicluster-data/ts-1/data/instance:
uuid: "4b04992c0da24371a1d3bc4bdd9102bf"
format_stamp: "Formatted at 2026-08-12 02:41:31 on dist-test-slave-mn4r"
server_key: "f8b50fc77c8183bdcf8374c2080a7b3c"
server_key_iv: "592b867ccbf976c7f3bb09b2d34cfaa3"
server_key_version: "encryptionkey@0"
I20260812 02:41:31.229323 31849 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502156971023-7698-0/minicluster-data/ts-1/wal/instance:
uuid: "4b04992c0da24371a1d3bc4bdd9102bf"
format_stamp: "Formatted at 2026-08-12 02:41:31 on dist-test-slave-mn4r"
server_key: "f8b50fc77c8183bdcf8374c2080a7b3c"
server_key_iv: "592b867ccbf976c7f3bb09b2d34cfaa3"
server_key_version: "encryptionkey@0"
I20260812 02:41:31.235176 31849 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260812 02:41:31.238727 31866 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:41:31.240118 31849 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 02:41:31.240290 31849 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502156971023-7698-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502156971023-7698-0/minicluster-data/ts-1/wal
uuid: "4b04992c0da24371a1d3bc4bdd9102bf"
format_stamp: "Formatted at 2026-08-12 02:41:31 on dist-test-slave-mn4r"
server_key: "f8b50fc77c8183bdcf8374c2080a7b3c"
server_key_iv: "592b867ccbf976c7f3bb09b2d34cfaa3"
server_key_version: "encryptionkey@0"
I20260812 02:41:31.240532 31849 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502156971023-7698-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502156971023-7698-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502156971023-7698-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:41:31.267086 31849 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:41:31.268000 31849 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:41:31.268325 31849 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:41:31.269383 31849 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:41:31.271003 31849 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:41:31.271099 31849 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:41:31.271175 31849 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:41:31.271255 31849 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:41:31.305780 31849 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.130:41409
I20260812 02:41:31.305873 32003 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.130:41409 every 8 connection(s)
I20260812 02:41:31.307163 31849 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502156971023-7698-0/minicluster-data/ts-1/data/info.pb
I20260812 02:41:31.312264 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 31849
I20260812 02:41:31.312455 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502156971023-7698-0/minicluster-data/ts-1/wal/instance
I20260812 02:41:31.312978 7698 external_mini_cluster.cc:1428] Setting key d29f25ed56aba997e5a95ee822205116
I20260812 02:41:31.315786 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502156971023-7698-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502156971023-7698-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502156971023-7698-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502156971023-7698-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.131:0
--local_ip_for_outbound_sockets=127.7.132.131
--webserver_interface=127.7.132.131
--webserver_port=0
--tserver_master_addrs=127.7.132.190:40105
--builtin_ntp_servers=127.7.132.148:32897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 02:41:31.320233 31845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:31.309846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:57603 (local address 127.7.132.190:40105)
0812 02:41:31.310045 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0812 02:41:31.310050 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:41:31.311518 (+ 1468us) server_negotiation.cc:407] Connection header received
0812 02:41:31.313676 (+ 2158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:41:31.313685 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:41:31.313775 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:41:31.313905 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:41:31.315072 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:31.315797 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:31.317456 (+ 1659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:31.317859 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:31.317989 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:31.319405 (+ 1416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:41:31.319409 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:41:31.319413 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:41:31.319431 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:41:31.319461 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:41:31.319473 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:41:31.319578 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0812 02:41:31.319819 (+ 241us) server_negotiation.cc:299] Negotiation successful
0812 02:41:31.319915 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260812 02:41:31.321230 32007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:31.310428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:40105 (local address 127.7.132.130:57603)
0812 02:41:31.311360 (+ 932us) negotiation.cc:107] Waiting for socket to connect
0812 02:41:31.311396 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 02:41:31.313386 (+ 1990us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:41:31.314212 (+ 826us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:41:31.314221 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:41:31.314308 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:41:31.314886 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:41:31.314902 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:41:31.316067 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:41:31.316071 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:41:31.317306 (+ 1235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:41:31.317318 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:41:31.318120 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:41:31.318128 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:41:31.318363 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:31.319085 (+ 722us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 02:41:31.319087 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:41:31.319126 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 02:41:31.319208 (+ 82us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:41:31.319576 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:41:31.319588 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:41:31.319599 (+ 11us) client_negotiation.cc:769] Sending connection context
0812 02:41:31.319784 (+ 185us) client_negotiation.cc:240] Negotiation successful
0812 02:41:31.320182 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":283,"threads_started":1}
I20260812 02:41:31.322748 32004 heartbeater.cc:347] Connected to a master server at 127.7.132.190:40105
I20260812 02:41:31.323055 32004 heartbeater.cc:464] Registering TS with master...
I20260812 02:41:31.323797 32004 heartbeater.cc:511] Master 127.7.132.190:40105 requested a full tablet report, sending...
I20260812 02:41:31.325314 31623 ts_manager.cc:194] Registered new tserver with Master: 4b04992c0da24371a1d3bc4bdd9102bf (127.7.132.130:41409)
I20260812 02:41:31.326534 31623 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.132.130:57603
W20260812 02:41:31.671862 32008 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:41:31.672150 32008 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:41:31.672237 32008 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:41:31.679495 32008 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:41:31.679595 32008 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:41:31.679700 32008 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.131
I20260812 02:41:31.689183 32008 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:32897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502156971023-7698-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502156971023-7698-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502156971023-7698-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.132.131
--webserver_port=0
--tserver_master_addrs=127.7.132.190:40105
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.131
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502156971023-7698-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:41:31.690474 32008 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:41:31.692245 32008 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:41:31.705785 32015 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:41:31.705832 32016 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:41:31.705861 32019 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:41:31.706842 32008 server_base.cc:1034] running on GCE node
I20260812 02:41:31.707425 32008 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:41:31.708557 32008 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:41:31.709800 32008 hybrid_clock.cc:648] HybridClock initialized: now 1786502491709711 us; error 77 us; skew 500 ppm
I20260812 02:41:31.710219 32008 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 02:41:31.712618 32008 webserver.cc:460] Webserver started at http://127.7.132.131:35021/ using document root <none> and password file <none>
I20260812 02:41:31.713423 32008 fs_manager.cc:359] Metadata directory not provided
I20260812 02:41:31.713554 32008 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:41:31.713896 32008 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:41:31.716902 32008 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502156971023-7698-0/minicluster-data/ts-2/data/instance:
uuid: "2f3ec6bcf6db44789ea5aa60225be5ea"
format_stamp: "Formatted at 2026-08-12 02:41:31 on dist-test-slave-mn4r"
server_key: "17192b5eb2a141fab0add14e9f0b0292"
server_key_iv: "111c9bbab6146af6f4c4678f1d4e7fa9"
server_key_version: "encryptionkey@0"
I20260812 02:41:31.717790 32008 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502156971023-7698-0/minicluster-data/ts-2/wal/instance:
uuid: "2f3ec6bcf6db44789ea5aa60225be5ea"
format_stamp: "Formatted at 2026-08-12 02:41:31 on dist-test-slave-mn4r"
server_key: "17192b5eb2a141fab0add14e9f0b0292"
server_key_iv: "111c9bbab6146af6f4c4678f1d4e7fa9"
server_key_version: "encryptionkey@0"
I20260812 02:41:31.723022 32008 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260812 02:41:31.726588 32027 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:41:31.727944 32008 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260812 02:41:31.728130 32008 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502156971023-7698-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502156971023-7698-0/minicluster-data/ts-2/wal
uuid: "2f3ec6bcf6db44789ea5aa60225be5ea"
format_stamp: "Formatted at 2026-08-12 02:41:31 on dist-test-slave-mn4r"
server_key: "17192b5eb2a141fab0add14e9f0b0292"
server_key_iv: "111c9bbab6146af6f4c4678f1d4e7fa9"
server_key_version: "encryptionkey@0"
I20260812 02:41:31.728322 32008 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502156971023-7698-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502156971023-7698-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502156971023-7698-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:41:31.745261 32008 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:41:31.746052 32008 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:41:31.746388 32008 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:41:31.747331 32008 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 02:41:31.748764 32008 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 02:41:31.748854 32008 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 02:41:31.748931 32008 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 02:41:31.748986 32008 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 02:41:31.782876 32008 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.131:43785
I20260812 02:41:31.782967 32160 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.131:43785 every 8 connection(s)
I20260812 02:41:31.784264 32008 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502156971023-7698-0/minicluster-data/ts-2/data/info.pb
I20260812 02:41:31.789233 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 32008
I20260812 02:41:31.789412 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502156971023-7698-0/minicluster-data/ts-2/wal/instance
I20260812 02:41:31.789891 7698 external_mini_cluster.cc:1428] Setting key 3d330174988b6bd09a87fb64b52128b8
I20260812 02:41:31.797746 31845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:31.787084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:41751 (local address 127.7.132.190:40105)
0812 02:41:31.787301 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0812 02:41:31.787305 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 02:41:31.788879 (+ 1574us) server_negotiation.cc:407] Connection header received
0812 02:41:31.791016 (+ 2137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:41:31.791030 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:41:31.791126 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:41:31.791234 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:41:31.792328 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:31.793235 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:31.794904 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:31.795345 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:31.795483 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:31.796896 (+ 1413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:41:31.796902 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:41:31.796905 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:41:31.796924 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:41:31.796958 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:41:31.796970 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:41:31.797075 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0812 02:41:31.797316 (+ 241us) server_negotiation.cc:299] Negotiation successful
0812 02:41:31.797417 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260812 02:41:31.798861 32165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:31.787702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:40105 (local address 127.7.132.131:41751)
0812 02:41:31.788703 (+ 1001us) negotiation.cc:107] Waiting for socket to connect
0812 02:41:31.788749 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 02:41:31.790750 (+ 2001us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:41:31.791481 (+ 731us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:41:31.791490 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:41:31.791585 (+ 95us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:41:31.792163 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:41:31.792179 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:41:31.793388 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:41:31.793398 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:41:31.794749 (+ 1351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:41:31.794763 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:41:31.795587 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:41:31.795595 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:41:31.795831 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:31.796554 (+ 723us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 02:41:31.796557 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:41:31.796598 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 02:41:31.796682 (+ 84us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:41:31.797097 (+ 415us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:41:31.797114 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:41:31.797120 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 02:41:31.797305 (+ 185us) client_negotiation.cc:240] Negotiation successful
0812 02:41:31.797725 (+ 420us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":313,"threads_started":1}
I20260812 02:41:31.800203 32161 heartbeater.cc:347] Connected to a master server at 127.7.132.190:40105
I20260812 02:41:31.800515 32161 heartbeater.cc:464] Registering TS with master...
I20260812 02:41:31.801303 32161 heartbeater.cc:511] Master 127.7.132.190:40105 requested a full tablet report, sending...
I20260812 02:41:31.802809 31623 ts_manager.cc:194] Registered new tserver with Master: 2f3ec6bcf6db44789ea5aa60225be5ea (127.7.132.131:43785)
I20260812 02:41:31.803735 31623 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.132.131:41751
I20260812 02:41:31.805157 7698 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 02:41:31.814975 31845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:31.809257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60652 (local address 127.7.132.190:40105)
0812 02:41:31.809434 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0812 02:41:31.809438 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 02:41:31.809960 (+ 522us) server_negotiation.cc:407] Connection header received
0812 02:41:31.810142 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:41:31.810147 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:41:31.810216 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:41:31.810298 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:41:31.810777 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:31.811473 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:31.812814 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:31.813298 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:31.813391 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:31.814410 (+ 1019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:41:31.814415 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:41:31.814418 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:41:31.814435 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:41:31.814469 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:41:31.814480 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:41:31.814552 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 02:41:31.814684 (+ 132us) server_negotiation.cc:299] Negotiation successful
0812 02:41:31.814755 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 02:41:31.818543 31623 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60652:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 02:41:31.825357 31623 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 02:41:31.853456 31843 heartbeater.cc:503] Master 127.7.132.190:40105 was elected leader, sending a full tablet report...
I20260812 02:41:31.864370 32176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:31.838210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.130:41409 (local address 127.0.0.1:36362)
0812 02:41:31.853367 (+ 15157us) negotiation.cc:107] Waiting for socket to connect
0812 02:41:31.853389 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 02:41:31.855800 (+ 2411us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:41:31.856911 (+ 1111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:41:31.856916 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:41:31.856947 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:41:31.857445 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:41:31.857461 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:41:31.859310 (+ 1849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:41:31.859316 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:41:31.861690 (+ 2374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:41:31.861712 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:41:31.863542 (+ 1830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:41:31.863548 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:41:31.863730 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:31.863770 (+ 40us) client_negotiation.cc:769] Sending connection context
0812 02:41:31.863867 (+ 97us) client_negotiation.cc:240] Negotiation successful
0812 02:41:31.864006 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14968,"mutex_wait_us":54,"thread_start_us":13981,"threads_started":1}
I20260812 02:41:31.864454 32179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:31.838561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36362 (local address 127.7.132.130:41409)
0812 02:41:31.851758 (+ 13197us) server_negotiation.cc:206] Beginning negotiation
0812 02:41:31.851771 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 02:41:31.853525 (+ 1754us) server_negotiation.cc:407] Connection header received
0812 02:41:31.856100 (+ 2575us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:41:31.856112 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:41:31.856405 (+ 293us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:41:31.856593 (+ 188us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:41:31.857671 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:31.859199 (+ 1528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:31.861918 (+ 2719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:31.863326 (+ 1408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:31.863493 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:31.863749 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0812 02:41:31.863930 (+ 181us) server_negotiation.cc:299] Negotiation successful
0812 02:41:31.864116 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12959,"thread_start_us":13223,"threads_started":1}
I20260812 02:41:31.864482 32178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:31.837848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.131:43785 (local address 127.0.0.1:50234)
0812 02:41:31.852636 (+ 14788us) negotiation.cc:107] Waiting for socket to connect
0812 02:41:31.852676 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 02:41:31.852930 (+ 254us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:41:31.855679 (+ 2749us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:41:31.855691 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:41:31.855746 (+ 55us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:41:31.856316 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:41:31.856339 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:41:31.858775 (+ 2436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:41:31.858781 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:41:31.861298 (+ 2517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:41:31.861311 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:41:31.863097 (+ 1786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:41:31.863104 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:41:31.863321 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:31.863389 (+ 68us) client_negotiation.cc:769] Sending connection context
0812 02:41:31.863494 (+ 105us) client_negotiation.cc:240] Negotiation successful
0812 02:41:31.863696 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14526,"mutex_wait_us":131,"thread_start_us":15710,"threads_started":1}
I20260812 02:41:31.865742 32180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:31.840171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:43127 (local address 127.0.0.1:44390)
0812 02:41:31.856700 (+ 16529us) negotiation.cc:107] Waiting for socket to connect
0812 02:41:31.856719 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 02:41:31.856843 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:41:31.858599 (+ 1756us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:41:31.858604 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:41:31.858631 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:41:31.859072 (+ 441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:41:31.859082 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:41:31.861095 (+ 2013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:41:31.861101 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:41:31.862801 (+ 1700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:41:31.862815 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:41:31.864907 (+ 2092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:41:31.864914 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:41:31.865103 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:31.865134 (+ 31us) client_negotiation.cc:769] Sending connection context
0812 02:41:31.865221 (+ 87us) client_negotiation.cc:240] Negotiation successful
0812 02:41:31.865355 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16395,"mutex_wait_us":24,"thread_start_us":16201,"threads_started":1}
I20260812 02:41:31.866530 32181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:31.840583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44390 (local address 127.7.132.129:43127)
0812 02:41:31.858135 (+ 17552us) server_negotiation.cc:206] Beginning negotiation
0812 02:41:31.858144 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 02:41:31.858171 (+ 27us) server_negotiation.cc:407] Connection header received
0812 02:41:31.858253 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:41:31.858258 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:41:31.858417 (+ 159us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:41:31.858615 (+ 198us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:41:31.859278 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:31.860922 (+ 1644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:31.863005 (+ 2083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:31.864296 (+ 1291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:31.864459 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:31.864756 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0812 02:41:31.866128 (+ 1372us) server_negotiation.cc:299] Negotiation successful
0812 02:41:31.866271 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17337,"thread_start_us":17183,"threads_started":1}
I20260812 02:41:31.868522 32177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:31.838048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50234 (local address 127.7.132.131:43785)
0812 02:41:31.854789 (+ 16741us) server_negotiation.cc:206] Beginning negotiation
0812 02:41:31.854802 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 02:41:31.854847 (+ 45us) server_negotiation.cc:407] Connection header received
0812 02:41:31.854977 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:41:31.854989 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:41:31.855349 (+ 360us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:41:31.856596 (+ 1247us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:41:31.856882 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:31.858601 (+ 1719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:31.861496 (+ 2895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:31.862869 (+ 1373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:31.865032 (+ 2163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:31.865322 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0812 02:41:31.865749 (+ 427us) server_negotiation.cc:299] Negotiation successful
0812 02:41:31.868202 (+ 2453us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16502,"thread_start_us":16308,"threads_started":1}
I20260812 02:41:31.868736 31925 tablet_service.cc:1467] Processing CreateTablet for tablet 8d5bf1752cdc4e189b1a91402d6c2a8f (DEFAULT_TABLE table=test-table [id=c72d49691b3f4c36875b3d40e3b28003]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:41:31.870326 31925 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d5bf1752cdc4e189b1a91402d6c2a8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:41:31.870010 31768 tablet_service.cc:1467] Processing CreateTablet for tablet 8d5bf1752cdc4e189b1a91402d6c2a8f (DEFAULT_TABLE table=test-table [id=c72d49691b3f4c36875b3d40e3b28003]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:41:31.871523 31768 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d5bf1752cdc4e189b1a91402d6c2a8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:41:31.872337 32084 tablet_service.cc:1467] Processing CreateTablet for tablet 8d5bf1752cdc4e189b1a91402d6c2a8f (DEFAULT_TABLE table=test-table [id=c72d49691b3f4c36875b3d40e3b28003]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 02:41:31.873754 32084 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d5bf1752cdc4e189b1a91402d6c2a8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:41:31.881692 32182 tablet_bootstrap.cc:492] T 8d5bf1752cdc4e189b1a91402d6c2a8f P 4b04992c0da24371a1d3bc4bdd9102bf: Bootstrap starting.
I20260812 02:41:31.885937 32182 tablet_bootstrap.cc:654] T 8d5bf1752cdc4e189b1a91402d6c2a8f P 4b04992c0da24371a1d3bc4bdd9102bf: Neither blocks nor log segments found. Creating new log.
I20260812 02:41:31.887007 32183 tablet_bootstrap.cc:492] T 8d5bf1752cdc4e189b1a91402d6c2a8f P 218fa25fae394acda26b0d08283fd9dd: Bootstrap starting.
I20260812 02:41:31.887475 32182 log.cc:824] T 8d5bf1752cdc4e189b1a91402d6c2a8f P 4b04992c0da24371a1d3bc4bdd9102bf: Log is configured to *not* fsync() on all Append() calls
I20260812 02:41:31.891774 32185 tablet_bootstrap.cc:492] T 8d5bf1752cdc4e189b1a91402d6c2a8f P 2f3ec6bcf6db44789ea5aa60225be5ea: Bootstrap starting.
I20260812 02:41:31.892092 32183 tablet_bootstrap.cc:654] T 8d5bf1752cdc4e189b1a91402d6c2a8f P 218fa25fae394acda26b0d08283fd9dd: Neither blocks nor log segments found. Creating new log.
I20260812 02:41:31.893329 32182 tablet_bootstrap.cc:492] T 8d5bf1752cdc4e189b1a91402d6c2a8f P 4b04992c0da24371a1d3bc4bdd9102bf: No bootstrap required, opened a new log
I20260812 02:41:31.893457 32183 log.cc:824] T 8d5bf1752cdc4e189b1a91402d6c2a8f P 218fa25fae394acda26b0d08283fd9dd: Log is configured to *not* fsync() on all Append() calls
I20260812 02:41:31.893630 32182 ts_tablet_manager.cc:1397] T 8d5bf1752cdc4e189b1a91402d6c2a8f P 4b04992c0da24371a1d3bc4bdd9102bf: Time spent bootstrapping tablet: real 0.012s user 0.009s sys 0.000s
I20260812 02:41:31.895027 32185 tablet_bootstrap.cc:654] T 8d5bf1752cdc4e189b1a91402d6c2a8f P 2f3ec6bcf6db44789ea5aa60225be5ea: Neither blocks nor log segments found. Creating new log.
I20260812 02:41:31.895990 32185 log.cc:824] T 8d5bf1752cdc4e189b1a91402d6c2a8f P 2f3ec6bcf6db44789ea5aa60225be5ea: Log is configured to *not* fsync() on all Append() calls
I20260812 02:41:31.896639 32183 tablet_bootstrap.cc:492] T 8d5bf1752cdc4e189b1a91402d6c2a8f P 218fa25fae394acda26b0d08283fd9dd: No bootstrap required, opened a new log
I20260812 02:41:31.896963 32183 ts_tablet_manager.cc:1397] T 8d5bf1752cdc4e189b1a91402d6c2a8f P 218fa25fae394acda26b0d08283fd9dd: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.004s
I20260812 02:41:31.898633 32185 tablet_bootstrap.cc:492] T 8d5bf1752cdc4e189b1a91402d6c2a8f P 2f3ec6bcf6db44789ea5aa60225be5ea: No bootstrap required, opened a new log
I20260812 02:41:31.898890 32185 ts_tablet_manager.cc:1397] T 8d5bf1752cdc4e189b1a91402d6c2a8f P 2f3ec6bcf6db44789ea5aa60225be5ea: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260812 02:41:31.904281 32182 raft_consensus.cc:348] T 8d5bf1752cdc4e189b1a91402d6c2a8f P 4b04992c0da24371a1d3bc4bdd9102bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f3ec6bcf6db44789ea5aa60225be5ea" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 43785 } } peers { permanent_uuid: "4b04992c0da24371a1d3bc4bdd9102bf" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 41409 } } peers { permanent_uuid: "218fa25fae394acda26b0d08283fd9dd" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 43127 } }
I20260812 02:41:31.904736 32182 raft_consensus.cc:374] T 8d5bf1752cdc4e189b1a91402d6c2a8f P 4b04992c0da24371a1d3bc4bdd9102bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:41:31.904877 32182 raft_consensus.cc:729] T 8d5bf1752cdc4e189b1a91402d6c2a8f P 4b04992c0da24371a1d3bc4bdd9102bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b04992c0da24371a1d3bc4bdd9102bf, State: Initialized, Role: FOLLOWER
I20260812 02:41:31.905539 32182 consensus_queue.cc:260] T 8d5bf1752cdc4e189b1a91402d6c2a8f P 4b04992c0da24371a1d3bc4bdd9102bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f3ec6bcf6db44789ea5aa60225be5ea" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 43785 } } peers { permanent_uuid: "4b04992c0da24371a1d3bc4bdd9102bf" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 41409 } } peers { permanent_uuid: "218fa25fae394acda26b0d08283fd9dd" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 43127 } }
I20260812 02:41:31.906724 32004 heartbeater.cc:503] Master 127.7.132.190:40105 was elected leader, sending a full tablet report...
I20260812 02:41:31.906981 32185 raft_consensus.cc:348] T 8d5bf1752cdc4e189b1a91402d6c2a8f P 2f3ec6bcf6db44789ea5aa60225be5ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f3ec6bcf6db44789ea5aa60225be5ea" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 43785 } } peers { permanent_uuid: "4b04992c0da24371a1d3bc4bdd9102bf" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 41409 } } peers { permanent_uuid: "218fa25fae394acda26b0d08283fd9dd" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 43127 } }
I20260812 02:41:31.907399 32185 raft_consensus.cc:374] T 8d5bf1752cdc4e189b1a91402d6c2a8f P 2f3ec6bcf6db44789ea5aa60225be5ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:41:31.907495 32185 raft_consensus.cc:729] T 8d5bf1752cdc4e189b1a91402d6c2a8f P 2f3ec6bcf6db44789ea5aa60225be5ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f3ec6bcf6db44789ea5aa60225be5ea, State: Initialized, Role: FOLLOWER
I20260812 02:41:31.907680 32182 ts_tablet_manager.cc:1428] T 8d5bf1752cdc4e189b1a91402d6c2a8f P 4b04992c0da24371a1d3bc4bdd9102bf: Time spent starting tablet: real 0.014s user 0.012s sys 0.004s
I20260812 02:41:31.907485 32183 raft_consensus.cc:348] T 8d5bf1752cdc4e189b1a91402d6c2a8f P 218fa25fae394acda26b0d08283fd9dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f3ec6bcf6db44789ea5aa60225be5ea" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 43785 } } peers { permanent_uuid: "4b04992c0da24371a1d3bc4bdd9102bf" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 41409 } } peers { permanent_uuid: "218fa25fae394acda26b0d08283fd9dd" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 43127 } }
I20260812 02:41:31.908116 32183 raft_consensus.cc:374] T 8d5bf1752cdc4e189b1a91402d6c2a8f P 218fa25fae394acda26b0d08283fd9dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:41:31.908056 32185 consensus_queue.cc:260] T 8d5bf1752cdc4e189b1a91402d6c2a8f P 2f3ec6bcf6db44789ea5aa60225be5ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f3ec6bcf6db44789ea5aa60225be5ea" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 43785 } } peers { permanent_uuid: "4b04992c0da24371a1d3bc4bdd9102bf" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 41409 } } peers { permanent_uuid: "218fa25fae394acda26b0d08283fd9dd" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 43127 } }
I20260812 02:41:31.908267 32183 raft_consensus.cc:729] T 8d5bf1752cdc4e189b1a91402d6c2a8f P 218fa25fae394acda26b0d08283fd9dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 218fa25fae394acda26b0d08283fd9dd, State: Initialized, Role: FOLLOWER
I20260812 02:41:31.908855 32183 consensus_queue.cc:260] T 8d5bf1752cdc4e189b1a91402d6c2a8f P 218fa25fae394acda26b0d08283fd9dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f3ec6bcf6db44789ea5aa60225be5ea" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 43785 } } peers { permanent_uuid: "4b04992c0da24371a1d3bc4bdd9102bf" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 41409 } } peers { permanent_uuid: "218fa25fae394acda26b0d08283fd9dd" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 43127 } }
I20260812 02:41:31.909708 32161 heartbeater.cc:503] Master 127.7.132.190:40105 was elected leader, sending a full tablet report...
I20260812 02:41:31.910349 32185 ts_tablet_manager.cc:1428] T 8d5bf1752cdc4e189b1a91402d6c2a8f P 2f3ec6bcf6db44789ea5aa60225be5ea: Time spent starting tablet: real 0.011s user 0.011s sys 0.001s
I20260812 02:41:31.918951 32183 ts_tablet_manager.cc:1428] T 8d5bf1752cdc4e189b1a91402d6c2a8f P 218fa25fae394acda26b0d08283fd9dd: Time spent starting tablet: real 0.022s user 0.013s sys 0.005s
W20260812 02:41:32.036088 32164 tablet.cc:2369] T 8d5bf1752cdc4e189b1a91402d6c2a8f P 2f3ec6bcf6db44789ea5aa60225be5ea: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 02:41:32.059334 32006 tablet.cc:2369] T 8d5bf1752cdc4e189b1a91402d6c2a8f P 4b04992c0da24371a1d3bc4bdd9102bf: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 02:41:32.081254 31844 tablet.cc:2369] T 8d5bf1752cdc4e189b1a91402d6c2a8f P 218fa25fae394acda26b0d08283fd9dd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 02:41:32.109609 32189 raft_consensus.cc:482] T 8d5bf1752cdc4e189b1a91402d6c2a8f P 4b04992c0da24371a1d3bc4bdd9102bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 02:41:32.109963 32189 raft_consensus.cc:504] T 8d5bf1752cdc4e189b1a91402d6c2a8f P 4b04992c0da24371a1d3bc4bdd9102bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f3ec6bcf6db44789ea5aa60225be5ea" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 43785 } } peers { permanent_uuid: "4b04992c0da24371a1d3bc4bdd9102bf" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 41409 } } peers { permanent_uuid: "218fa25fae394acda26b0d08283fd9dd" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 43127 } }
I20260812 02:41:32.111876 32189 leader_election.cc:288] T 8d5bf1752cdc4e189b1a91402d6c2a8f P 4b04992c0da24371a1d3bc4bdd9102bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2f3ec6bcf6db44789ea5aa60225be5ea (127.7.132.131:43785), 218fa25fae394acda26b0d08283fd9dd (127.7.132.129:43127)
I20260812 02:41:32.119374 32181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:32.112538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:59365 (local address 127.7.132.129:43127)
0812 02:41:32.112911 (+ 373us) server_negotiation.cc:206] Beginning negotiation
0812 02:41:32.112917 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:41:32.113776 (+ 859us) server_negotiation.cc:407] Connection header received
0812 02:41:32.113948 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:41:32.113956 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:41:32.114056 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:41:32.114183 (+ 127us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:41:32.114875 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:32.115604 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:32.117500 (+ 1896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:32.118373 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:32.118553 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:32.118635 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 02:41:32.118976 (+ 341us) server_negotiation.cc:299] Negotiation successful
0812 02:41:32.119113 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141}
I20260812 02:41:32.119371 32177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:32.112362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:47257 (local address 127.7.132.131:43785)
0812 02:41:32.112652 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0812 02:41:32.112661 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 02:41:32.113607 (+ 946us) server_negotiation.cc:407] Connection header received
0812 02:41:32.113776 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:41:32.113781 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:41:32.113869 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:41:32.113959 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 02:41:32.114702 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:32.115468 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:32.117530 (+ 2062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:32.118382 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:32.118595 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:32.118680 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 02:41:32.119014 (+ 334us) server_negotiation.cc:299] Negotiation successful
0812 02:41:32.119140 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260812 02:41:32.119519 32194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:32.112092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.131:43785 (local address 127.7.132.130:47257)
0812 02:41:32.113474 (+ 1382us) negotiation.cc:107] Waiting for socket to connect
0812 02:41:32.113494 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 02:41:32.113632 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:41:32.114007 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:41:32.114014 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:41:32.114050 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:41:32.114540 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:41:32.114549 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:41:32.115604 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:41:32.115610 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:41:32.117282 (+ 1672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:41:32.117294 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:41:32.118604 (+ 1310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:41:32.118610 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:41:32.118831 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:32.118855 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 02:41:32.118966 (+ 111us) client_negotiation.cc:240] Negotiation successful
0812 02:41:32.119113 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1260,"mutex_wait_us":121,"thread_start_us":1084,"threads_started":1}
I20260812 02:41:32.119542 32192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:32.112249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:43127 (local address 127.7.132.130:59365)
0812 02:41:32.113609 (+ 1360us) negotiation.cc:107] Waiting for socket to connect
0812 02:41:32.113630 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 02:41:32.113796 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:41:32.114208 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:41:32.114211 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:41:32.114247 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 02:41:32.114705 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:41:32.114717 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:41:32.115747 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:41:32.115751 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:41:32.117281 (+ 1530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:41:32.117294 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:41:32.118599 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:41:32.118606 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:41:32.118806 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:32.118832 (+ 26us) client_negotiation.cc:769] Sending connection context
0812 02:41:32.118907 (+ 75us) client_negotiation.cc:240] Negotiation successful
0812 02:41:32.119062 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1220,"mutex_wait_us":37,"thread_start_us":622,"threads_started":1}
I20260812 02:41:32.120790 32107 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d5bf1752cdc4e189b1a91402d6c2a8f" candidate_uuid: "4b04992c0da24371a1d3bc4bdd9102bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f3ec6bcf6db44789ea5aa60225be5ea" is_pre_election: true
I20260812 02:41:32.120905 31791 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d5bf1752cdc4e189b1a91402d6c2a8f" candidate_uuid: "4b04992c0da24371a1d3bc4bdd9102bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "218fa25fae394acda26b0d08283fd9dd" is_pre_election: true
I20260812 02:41:32.121369 32107 raft_consensus.cc:2454] T 8d5bf1752cdc4e189b1a91402d6c2a8f P 2f3ec6bcf6db44789ea5aa60225be5ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b04992c0da24371a1d3bc4bdd9102bf in term 0.
I20260812 02:41:32.121430 31791 raft_consensus.cc:2454] T 8d5bf1752cdc4e189b1a91402d6c2a8f P 218fa25fae394acda26b0d08283fd9dd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b04992c0da24371a1d3bc4bdd9102bf in term 0.
I20260812 02:41:32.122074 31868 leader_election.cc:302] T 8d5bf1752cdc4e189b1a91402d6c2a8f P 4b04992c0da24371a1d3bc4bdd9102bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f3ec6bcf6db44789ea5aa60225be5ea, 4b04992c0da24371a1d3bc4bdd9102bf; no voters:
I20260812 02:41:32.122567 32189 raft_consensus.cc:2781] T 8d5bf1752cdc4e189b1a91402d6c2a8f P 4b04992c0da24371a1d3bc4bdd9102bf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 02:41:32.122735 32189 raft_consensus.cc:482] T 8d5bf1752cdc4e189b1a91402d6c2a8f P 4b04992c0da24371a1d3bc4bdd9102bf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 02:41:32.122855 32189 raft_consensus.cc:3037] T 8d5bf1752cdc4e189b1a91402d6c2a8f P 4b04992c0da24371a1d3bc4bdd9102bf [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:41:32.125114 32189 raft_consensus.cc:504] T 8d5bf1752cdc4e189b1a91402d6c2a8f P 4b04992c0da24371a1d3bc4bdd9102bf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f3ec6bcf6db44789ea5aa60225be5ea" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 43785 } } peers { permanent_uuid: "4b04992c0da24371a1d3bc4bdd9102bf" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 41409 } } peers { permanent_uuid: "218fa25fae394acda26b0d08283fd9dd" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 43127 } }
I20260812 02:41:32.125769 32189 leader_election.cc:288] T 8d5bf1752cdc4e189b1a91402d6c2a8f P 4b04992c0da24371a1d3bc4bdd9102bf [CANDIDATE]: Term 1 election: Requested vote from peers 2f3ec6bcf6db44789ea5aa60225be5ea (127.7.132.131:43785), 218fa25fae394acda26b0d08283fd9dd (127.7.132.129:43127)
I20260812 02:41:32.126305 32107 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d5bf1752cdc4e189b1a91402d6c2a8f" candidate_uuid: "4b04992c0da24371a1d3bc4bdd9102bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f3ec6bcf6db44789ea5aa60225be5ea"
I20260812 02:41:32.126325 31791 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d5bf1752cdc4e189b1a91402d6c2a8f" candidate_uuid: "4b04992c0da24371a1d3bc4bdd9102bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "218fa25fae394acda26b0d08283fd9dd"
I20260812 02:41:32.126549 31791 raft_consensus.cc:3037] T 8d5bf1752cdc4e189b1a91402d6c2a8f P 218fa25fae394acda26b0d08283fd9dd [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:41:32.126549 32107 raft_consensus.cc:3037] T 8d5bf1752cdc4e189b1a91402d6c2a8f P 2f3ec6bcf6db44789ea5aa60225be5ea [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:41:32.129948 31791 raft_consensus.cc:2454] T 8d5bf1752cdc4e189b1a91402d6c2a8f P 218fa25fae394acda26b0d08283fd9dd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b04992c0da24371a1d3bc4bdd9102bf in term 1.
I20260812 02:41:32.129948 32107 raft_consensus.cc:2454] T 8d5bf1752cdc4e189b1a91402d6c2a8f P 2f3ec6bcf6db44789ea5aa60225be5ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b04992c0da24371a1d3bc4bdd9102bf in term 1.
I20260812 02:41:32.130517 31868 leader_election.cc:302] T 8d5bf1752cdc4e189b1a91402d6c2a8f P 4b04992c0da24371a1d3bc4bdd9102bf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 218fa25fae394acda26b0d08283fd9dd, 2f3ec6bcf6db44789ea5aa60225be5ea, 4b04992c0da24371a1d3bc4bdd9102bf; no voters:
I20260812 02:41:32.130854 32189 raft_consensus.cc:2781] T 8d5bf1752cdc4e189b1a91402d6c2a8f P 4b04992c0da24371a1d3bc4bdd9102bf [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:41:32.131289 32189 raft_consensus.cc:686] T 8d5bf1752cdc4e189b1a91402d6c2a8f P 4b04992c0da24371a1d3bc4bdd9102bf [term 1 LEADER]: Becoming Leader. State: Replica: 4b04992c0da24371a1d3bc4bdd9102bf, State: Running, Role: LEADER
I20260812 02:41:32.131834 32189 consensus_queue.cc:237] T 8d5bf1752cdc4e189b1a91402d6c2a8f P 4b04992c0da24371a1d3bc4bdd9102bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f3ec6bcf6db44789ea5aa60225be5ea" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 43785 } } peers { permanent_uuid: "4b04992c0da24371a1d3bc4bdd9102bf" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 41409 } } peers { permanent_uuid: "218fa25fae394acda26b0d08283fd9dd" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 43127 } }
I20260812 02:41:32.136605 31622 catalog_manager.cc:5495] T 8d5bf1752cdc4e189b1a91402d6c2a8f P 4b04992c0da24371a1d3bc4bdd9102bf reported cstate change: term changed from 0 to 1, leader changed from <none> to 4b04992c0da24371a1d3bc4bdd9102bf (127.7.132.130). New cstate: current_term: 1 leader_uuid: "4b04992c0da24371a1d3bc4bdd9102bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f3ec6bcf6db44789ea5aa60225be5ea" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 43785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b04992c0da24371a1d3bc4bdd9102bf" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 41409 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "218fa25fae394acda26b0d08283fd9dd" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 43127 } health_report { overall_health: UNKNOWN } } }
I20260812 02:41:32.158530 32179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:32.152932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36374 (local address 127.7.132.130:41409)
0812 02:41:32.153164 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0812 02:41:32.153171 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:41:32.153287 (+ 116us) server_negotiation.cc:407] Connection header received
0812 02:41:32.153527 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:41:32.153531 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:41:32.153633 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:41:32.153785 (+ 152us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 02:41:32.154192 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:32.154889 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:32.156297 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:32.156722 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:32.156867 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:32.157357 (+ 490us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 02:41:32.157871 (+ 514us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 02:41:32.158040 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0812 02:41:32.158162 (+ 122us) server_negotiation.cc:299] Negotiation successful
0812 02:41:32.158246 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260812 02:41:32.166823 31791 raft_consensus.cc:1261] T 8d5bf1752cdc4e189b1a91402d6c2a8f P 218fa25fae394acda26b0d08283fd9dd [term 1 FOLLOWER]: Refusing update from remote peer 4b04992c0da24371a1d3bc4bdd9102bf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:41:32.166878 32107 raft_consensus.cc:1261] T 8d5bf1752cdc4e189b1a91402d6c2a8f P 2f3ec6bcf6db44789ea5aa60225be5ea [term 1 FOLLOWER]: Refusing update from remote peer 4b04992c0da24371a1d3bc4bdd9102bf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:41:32.167876 32196 consensus_queue.cc:1035] T 8d5bf1752cdc4e189b1a91402d6c2a8f P 4b04992c0da24371a1d3bc4bdd9102bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f3ec6bcf6db44789ea5aa60225be5ea" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 43785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:41:32.168229 32189 consensus_queue.cc:1035] T 8d5bf1752cdc4e189b1a91402d6c2a8f P 4b04992c0da24371a1d3bc4bdd9102bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "218fa25fae394acda26b0d08283fd9dd" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 43127 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:41:32.179584 32203 mvcc.cc:205] Tried to move back new op lower bound from 7317514207905443840 to 7317514207776350208. Current Snapshot: MvccSnapshot[applied={T|T < 7317514207905443840}]
I20260812 02:41:32.192238 31622 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60652:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 02:41:32.192559 31622 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60652:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 02:41:32.198181 31622 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e768c0fc1ae946cb879ef98dbf3d6e30: Sending DeleteTablet for 3 replicas of tablet 8d5bf1752cdc4e189b1a91402d6c2a8f
I20260812 02:41:32.198521 32202 mvcc.cc:205] Tried to move back new op lower bound from 7317514207905443840 to 7317514207776350208. Current Snapshot: MvccSnapshot[applied={T|T < 7317514207905443840}]
I20260812 02:41:32.200202 32084 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d5bf1752cdc4e189b1a91402d6c2a8f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:41:32 UTC) from {username='slave'} at 127.0.0.1:50234
I20260812 02:41:32.200783 31768 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d5bf1752cdc4e189b1a91402d6c2a8f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:41:32 UTC) from {username='slave'} at 127.0.0.1:44390
I20260812 02:41:32.202673 31925 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d5bf1752cdc4e189b1a91402d6c2a8f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 02:41:32 UTC) from {username='slave'} at 127.0.0.1:36362
I20260812 02:41:32.212481 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 31697
I20260812 02:41:32.215173 32214 tablet_replica.cc:330] T 8d5bf1752cdc4e189b1a91402d6c2a8f P 218fa25fae394acda26b0d08283fd9dd: stopping tablet replica
I20260812 02:41:32.216045 32215 tablet_replica.cc:330] T 8d5bf1752cdc4e189b1a91402d6c2a8f P 4b04992c0da24371a1d3bc4bdd9102bf: stopping tablet replica
I20260812 02:41:32.216501 32215 raft_consensus.cc:2229] T 8d5bf1752cdc4e189b1a91402d6c2a8f P 4b04992c0da24371a1d3bc4bdd9102bf [term 1 LEADER]: Raft consensus shutting down.
I20260812 02:41:32.217645 32214 raft_consensus.cc:2229] T 8d5bf1752cdc4e189b1a91402d6c2a8f P 218fa25fae394acda26b0d08283fd9dd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 02:41:32.218056 32214 raft_consensus.cc:2258] T 8d5bf1752cdc4e189b1a91402d6c2a8f P 218fa25fae394acda26b0d08283fd9dd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:41:32.219383 32213 tablet_replica.cc:330] T 8d5bf1752cdc4e189b1a91402d6c2a8f P 2f3ec6bcf6db44789ea5aa60225be5ea: stopping tablet replica
I20260812 02:41:32.219798 32213 raft_consensus.cc:2229] T 8d5bf1752cdc4e189b1a91402d6c2a8f P 2f3ec6bcf6db44789ea5aa60225be5ea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 02:41:32.220296 32213 raft_consensus.cc:2258] T 8d5bf1752cdc4e189b1a91402d6c2a8f P 2f3ec6bcf6db44789ea5aa60225be5ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:41:32.220769 32215 raft_consensus.cc:2258] T 8d5bf1752cdc4e189b1a91402d6c2a8f P 4b04992c0da24371a1d3bc4bdd9102bf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 02:41:32.221844 32214 ts_tablet_manager.cc:1905] T 8d5bf1752cdc4e189b1a91402d6c2a8f P 218fa25fae394acda26b0d08283fd9dd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:41:32.222767 32213 ts_tablet_manager.cc:1905] T 8d5bf1752cdc4e189b1a91402d6c2a8f P 2f3ec6bcf6db44789ea5aa60225be5ea: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:41:32.223346 32181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:32.213964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44400 (local address 127.7.132.129:43127)
0812 02:41:32.214254 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0812 02:41:32.214265 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 02:41:32.214692 (+ 427us) server_negotiation.cc:407] Connection header received
0812 02:41:32.214887 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:41:32.214897 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:41:32.215012 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:41:32.215147 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:41:32.215650 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:32.216830 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:32.219217 (+ 2387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:32.219859 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:32.219987 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:32.221535 (+ 1548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:41:32.221547 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:41:32.221562 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:41:32.222381 (+ 819us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:41:32.222456 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:41:32.222489 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:41:32.222578 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0812 02:41:32.222833 (+ 255us) server_negotiation.cc:299] Negotiation successful
0812 02:41:32.222951 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260812 02:41:32.224720 31838 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:41:32.228760 32213 ts_tablet_manager.cc:1918] T 8d5bf1752cdc4e189b1a91402d6c2a8f P 2f3ec6bcf6db44789ea5aa60225be5ea: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 02:41:32.228950 32213 log.cc:1196] T 8d5bf1752cdc4e189b1a91402d6c2a8f P 2f3ec6bcf6db44789ea5aa60225be5ea: Deleting WAL directory at /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502156971023-7698-0/minicluster-data/ts-2/wal/wals/8d5bf1752cdc4e189b1a91402d6c2a8f
I20260812 02:41:32.229410 32213 ts_tablet_manager.cc:1939] T 8d5bf1752cdc4e189b1a91402d6c2a8f P 2f3ec6bcf6db44789ea5aa60225be5ea: Deleting consensus metadata
I20260812 02:41:32.233404 32215 ts_tablet_manager.cc:1905] T 8d5bf1752cdc4e189b1a91402d6c2a8f P 4b04992c0da24371a1d3bc4bdd9102bf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 02:41:32.238648 31604 catalog_manager.cc:4841] TS 2f3ec6bcf6db44789ea5aa60225be5ea (127.7.132.131:43785): tablet 8d5bf1752cdc4e189b1a91402d6c2a8f (table test-table [id=c72d49691b3f4c36875b3d40e3b28003]) successfully deleted
I20260812 02:41:32.242276 32215 ts_tablet_manager.cc:1918] T 8d5bf1752cdc4e189b1a91402d6c2a8f P 4b04992c0da24371a1d3bc4bdd9102bf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 02:41:32.242516 32215 log.cc:1196] T 8d5bf1752cdc4e189b1a91402d6c2a8f P 4b04992c0da24371a1d3bc4bdd9102bf: Deleting WAL directory at /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786502156971023-7698-0/minicluster-data/ts-1/wal/wals/8d5bf1752cdc4e189b1a91402d6c2a8f
I20260812 02:41:32.243201 32215 ts_tablet_manager.cc:1939] T 8d5bf1752cdc4e189b1a91402d6c2a8f P 4b04992c0da24371a1d3bc4bdd9102bf: Deleting consensus metadata
I20260812 02:41:32.245447 31608 catalog_manager.cc:4841] TS 4b04992c0da24371a1d3bc4bdd9102bf (127.7.132.130:41409): tablet 8d5bf1752cdc4e189b1a91402d6c2a8f (table test-table [id=c72d49691b3f4c36875b3d40e3b28003]) successfully deleted
I20260812 02:41:32.335039 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 31697
W20260812 02:41:32.359367 31603 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.132.129:43127 (error 108)
I20260812 02:41:32.360190 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 31849
I20260812 02:41:32.361536 32180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:32.360875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:43127 (local address 127.0.0.1:44408)
0812 02:41:32.361172 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0812 02:41:32.361314 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.129:43127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167}
W20260812 02:41:32.361819 31603 catalog_manager.cc:4584] TS 218fa25fae394acda26b0d08283fd9dd (127.7.132.129:43127): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8d5bf1752cdc4e189b1a91402d6c2a8f: Network error: Client connection negotiation failed: client connection to 127.7.132.129:43127: connect: Connection refused (error 111)
I20260812 02:41:32.368562 32179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:32.361084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36384 (local address 127.7.132.130:41409)
0812 02:41:32.361283 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0812 02:41:32.361290 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:41:32.361316 (+ 26us) server_negotiation.cc:407] Connection header received
0812 02:41:32.361494 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:41:32.361499 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:41:32.361612 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:41:32.361733 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:41:32.362353 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:32.363172 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:32.365123 (+ 1951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:32.365668 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:32.365811 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:32.367052 (+ 1241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:41:32.367062 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:41:32.367076 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:41:32.367740 (+ 664us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:41:32.367787 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:41:32.367811 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:41:32.367942 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0812 02:41:32.368082 (+ 140us) server_negotiation.cc:299] Negotiation successful
0812 02:41:32.368202 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260812 02:41:32.369416 31999 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:41:32.415606 32180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:32.415119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:43127 (local address 127.0.0.1:44422)
0812 02:41:32.415328 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0812 02:41:32.415429 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.129:43127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260812 02:41:32.416733 32180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:32.416392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:43127 (local address 127.0.0.1:44428)
0812 02:41:32.416533 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0812 02:41:32.416605 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.129:43127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260812 02:41:32.477444 32180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:32.476872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:43127 (local address 127.0.0.1:44434)
0812 02:41:32.477127 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0812 02:41:32.477229 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.129:43127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110}
I20260812 02:41:32.478585 32180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:32.478251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:43127 (local address 127.0.0.1:44438)
0812 02:41:32.478381 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0812 02:41:32.478467 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.129:43127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260812 02:41:32.483702 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 31849
I20260812 02:41:32.508334 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 32008
I20260812 02:41:32.515955 32177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:32.509389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50242 (local address 127.7.132.131:43785)
0812 02:41:32.509594 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0812 02:41:32.509599 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:41:32.509620 (+ 21us) server_negotiation.cc:407] Connection header received
0812 02:41:32.509840 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:41:32.509845 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:41:32.509959 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:41:32.510125 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:41:32.510683 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:32.511464 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:32.512777 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:32.513194 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:32.513374 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:32.514471 (+ 1097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:41:32.514481 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:41:32.514497 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:41:32.515084 (+ 587us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:41:32.515136 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:41:32.515157 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:41:32.515250 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0812 02:41:32.515456 (+ 206us) server_negotiation.cc:299] Negotiation successful
0812 02:41:32.515597 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260812 02:41:32.516659 32154 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:41:32.559435 32180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:32.558916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:43127 (local address 127.0.0.1:44448)
0812 02:41:32.559151 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0812 02:41:32.559251 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.129:43127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260812 02:41:32.560595 32180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:32.560166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:43127 (local address 127.0.0.1:44450)
0812 02:41:32.560377 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0812 02:41:32.560482 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.129:43127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103}
I20260812 02:41:32.608523 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 32008
I20260812 02:41:32.630239 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 31586
I20260812 02:41:32.631346 31660 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:41:32.741932 32180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:32.741242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:43127 (local address 127.0.0.1:44458)
0812 02:41:32.741465 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0812 02:41:32.741627 (+ 162us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.129:43127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260812 02:41:32.742055 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 31586
2026-08-12T02:41:32Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (3038 ms)
[----------] 6 tests from EncryptionPolicyTest (23920 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786502156971023-7698-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 02:41:32 dist-test-slave-mn4r krb5kdc[32228](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 02:41:32 dist-test-slave-mn4r krb5kdc[32228](info): set up 2 sockets
Aug 12 02:41:32 dist-test-slave-mn4r krb5kdc[32228](info): commencing operation
krb5kdc: starting...
W20260812 02:41:33.993279 7698 mini_kdc.cc:121] Time spent starting KDC: real 1.202s user 0.000s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786502156971023-7698-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786502156971023-7698-0/krb5kdc/test-user.keytab.
Aug 12 02:41:34 dist-test-slave-mn4r krb5kdc[32228](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502494, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T02:41:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T02:41:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786502156971023-7698-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786502156971023-7698-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 02:41:34.322352 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786502156971023-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786502156971023-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786502156971023-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34533
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786502156971023-7698-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45217
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34533
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:41:34.670876 32243 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:41:34.671188 32243 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:41:34.671293 32243 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:41:34.678596 32243 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:41:34.678707 32243 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:41:34.678752 32243 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:41:34.678814 32243 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:41:34.678858 32243 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:41:34.688491 32243 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45217
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786502156971023-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786502156971023-7698-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34533
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786502156971023-7698-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34533
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786502156971023-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:41:34.689844 32243 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:41:34.691224 32243 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:41:34.703364 32252 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:41:34.703454 32249 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:41:34.703408 32248 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:41:34.704715 32243 server_base.cc:1034] running on GCE node
I20260812 02:41:34.705472 32243 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:41:34.706955 32243 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:41:34.708165 32243 hybrid_clock.cc:648] HybridClock initialized: now 1786502494708116 us; error 34 us; skew 500 ppm
I20260812 02:41:34.708525 32243 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:41:34 dist-test-slave-mn4r krb5kdc[32228](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502494, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:41:34.718639 32243 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 02:41:34.720288 32243 webserver.cc:460] Webserver started at http://127.0.0.1:42457/ using document root <none> and password file <none>
I20260812 02:41:34.721043 32243 fs_manager.cc:359] Metadata directory not provided
I20260812 02:41:34.721182 32243 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:41:34.721518 32243 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:41:34.724809 32243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786502156971023-7698-0/minicluster-data/master-0/data/instance:
uuid: "9087617cd7814554a24295a4f8e83acc"
format_stamp: "Formatted at 2026-08-12 02:41:34 on dist-test-slave-mn4r"
server_key: "382bc85089df8ec2f251aa935ae32fd6"
server_key_iv: "94748fb6204dedd0367d7bc0342e9083"
server_key_version: "encryptionkey@0"
I20260812 02:41:34.725628 32243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786502156971023-7698-0/minicluster-data/master-0/wal/instance:
uuid: "9087617cd7814554a24295a4f8e83acc"
format_stamp: "Formatted at 2026-08-12 02:41:34 on dist-test-slave-mn4r"
server_key: "382bc85089df8ec2f251aa935ae32fd6"
server_key_iv: "94748fb6204dedd0367d7bc0342e9083"
server_key_version: "encryptionkey@0"
I20260812 02:41:34.730953 32243 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260812 02:41:34.734534 32259 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:41:34.735846 32243 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260812 02:41:34.736037 32243 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786502156971023-7698-0/minicluster-data/master-0/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786502156971023-7698-0/minicluster-data/master-0/wal
uuid: "9087617cd7814554a24295a4f8e83acc"
format_stamp: "Formatted at 2026-08-12 02:41:34 on dist-test-slave-mn4r"
server_key: "382bc85089df8ec2f251aa935ae32fd6"
server_key_iv: "94748fb6204dedd0367d7bc0342e9083"
server_key_version: "encryptionkey@0"
I20260812 02:41:34.736241 32243 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786502156971023-7698-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786502156971023-7698-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786502156971023-7698-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:41:34.769129 32243 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:41:34.773552 32243 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:41:34.773885 32243 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:41:34.793412 32243 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34533
I20260812 02:41:34.793445 32320 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34533 every 8 connection(s)
I20260812 02:41:34.794893 32243 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
I20260812 02:41:34.798789 32322 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:41:34.804880 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 32243
I20260812 02:41:34.805136 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786502156971023-7698-0/minicluster-data/master-0/wal/instance
I20260812 02:41:34.805979 7698 external_mini_cluster.cc:1428] Setting key 1201e27aa3f5a4e8d87b80b970c905fc
I20260812 02:41:34.810508 32322 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9087617cd7814554a24295a4f8e83acc: Bootstrap starting.
I20260812 02:41:34.814391 32322 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9087617cd7814554a24295a4f8e83acc: Neither blocks nor log segments found. Creating new log.
I20260812 02:41:34.815625 32322 log.cc:824] T 00000000000000000000000000000000 P 9087617cd7814554a24295a4f8e83acc: Log is configured to *not* fsync() on all Append() calls
I20260812 02:41:34.818936 32322 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9087617cd7814554a24295a4f8e83acc: No bootstrap required, opened a new log
Aug 12 02:41:34 dist-test-slave-mn4r krb5kdc[32228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502494, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 02:41:34.827718 32322 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9087617cd7814554a24295a4f8e83acc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9087617cd7814554a24295a4f8e83acc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34533 } }
I20260812 02:41:34.828131 32322 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9087617cd7814554a24295a4f8e83acc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:41:34.828269 32322 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9087617cd7814554a24295a4f8e83acc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9087617cd7814554a24295a4f8e83acc, State: Initialized, Role: FOLLOWER
I20260812 02:41:34.829048 32322 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9087617cd7814554a24295a4f8e83acc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9087617cd7814554a24295a4f8e83acc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34533 } }
I20260812 02:41:34.829391 32322 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9087617cd7814554a24295a4f8e83acc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 02:41:34.829605 32322 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9087617cd7814554a24295a4f8e83acc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 02:41:34.829823 32322 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9087617cd7814554a24295a4f8e83acc [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:41:34.830883 32326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:34.808120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46233 (local address 127.0.0.1:34533)
0812 02:41:34.808945 (+ 825us) server_negotiation.cc:206] Beginning negotiation
0812 02:41:34.808957 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 02:41:34.808997 (+ 40us) server_negotiation.cc:407] Connection header received
0812 02:41:34.810782 (+ 1785us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:41:34.810805 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:41:34.811188 (+ 383us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:41:34.811497 (+ 309us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:41:34.813447 (+ 1950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:34.814919 (+ 1472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:34.816706 (+ 1787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:34.817478 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:34.817608 (+ 130us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:34.822781 (+ 5173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:41:34.822791 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:41:34.822807 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:41:34.822858 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:41:34.826026 (+ 3168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:41:34.826944 (+ 918us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:41:34.826951 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:41:34.826958 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:41:34.827153 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:41:34.827653 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:41:34.827660 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:41:34.827668 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:41:34.828315 (+ 647us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:41:34.828632 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0812 02:41:34.829195 (+ 563us) server_negotiation.cc:299] Negotiation successful
0812 02:41:34.829568 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":618,"thread_start_us":411,"threads_started":1}
I20260812 02:41:34.832401 32322 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9087617cd7814554a24295a4f8e83acc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9087617cd7814554a24295a4f8e83acc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34533 } }
I20260812 02:41:34.832927 32322 leader_election.cc:302] T 00000000000000000000000000000000 P 9087617cd7814554a24295a4f8e83acc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9087617cd7814554a24295a4f8e83acc; no voters:
I20260812 02:41:34.833745 32322 leader_election.cc:288] T 00000000000000000000000000000000 P 9087617cd7814554a24295a4f8e83acc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 02:41:34.833884 32328 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9087617cd7814554a24295a4f8e83acc [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:41:34.835181 32328 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9087617cd7814554a24295a4f8e83acc [term 1 LEADER]: Becoming Leader. State: Replica: 9087617cd7814554a24295a4f8e83acc, State: Running, Role: LEADER
I20260812 02:41:34.835537 7698 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 02:41:34.835843 32328 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9087617cd7814554a24295a4f8e83acc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9087617cd7814554a24295a4f8e83acc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34533 } }
I20260812 02:41:34.836212 32322 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9087617cd7814554a24295a4f8e83acc [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:41:34.842561 32330 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9087617cd7814554a24295a4f8e83acc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9087617cd7814554a24295a4f8e83acc. Latest consensus state: current_term: 1 leader_uuid: "9087617cd7814554a24295a4f8e83acc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9087617cd7814554a24295a4f8e83acc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34533 } } }
I20260812 02:41:34.843185 32330 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9087617cd7814554a24295a4f8e83acc [sys.catalog]: This master's current role is: LEADER
I20260812 02:41:34.842514 32329 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9087617cd7814554a24295a4f8e83acc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9087617cd7814554a24295a4f8e83acc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9087617cd7814554a24295a4f8e83acc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34533 } } }
I20260812 02:41:34.843596 32329 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9087617cd7814554a24295a4f8e83acc [sys.catalog]: This master's current role is: LEADER
I20260812 02:41:34.844792 32342 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:41:34.852180 32342 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 02:41:34.867015 32342 catalog_manager.cc:1347] Generated new cluster ID: b11a69bd50e444d596472e2e8099026f
I20260812 02:41:34.867266 32342 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 02:41:34.876505 32326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:34.847730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48327 (local address 127.0.0.1:34533)
0812 02:41:34.847944 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0812 02:41:34.847954 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 02:41:34.858059 (+ 10105us) server_negotiation.cc:407] Connection header received
0812 02:41:34.858332 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:41:34.858338 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:41:34.858435 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:41:34.858566 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:41:34.860814 (+ 2248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:34.862117 (+ 1303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:34.864000 (+ 1883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:34.864668 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:34.864772 (+ 104us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:34.868500 (+ 3728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:41:34.868507 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:41:34.868512 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:41:34.868559 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:41:34.872487 (+ 3928us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:41:34.873545 (+ 1058us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:41:34.873555 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:41:34.873559 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:41:34.873760 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:41:34.874410 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:41:34.874415 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:41:34.874419 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:41:34.874869 (+ 450us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:41:34.875150 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0812 02:41:34.875927 (+ 777us) server_negotiation.cc:299] Negotiation successful
0812 02:41:34.876126 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260812 02:41:34.902281 32342 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 02:41:34.903676 32342 catalog_manager.cc:1504] Loading token signing keys...
I20260812 02:41:34.910640 32342 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9087617cd7814554a24295a4f8e83acc: Generated new TSK 0
I20260812 02:41:34.911477 32342 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 02:41:34.918465 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 32243
I20260812 02:41:34.919932 32315 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:41:35.014806 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 32243
W20260812 02:41:35.038632 32226 connection.cc:535] client connection to 127.0.0.1:34533 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34533): Connection reset by peer (error 104)
2026-08-12T02:41:35Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (2267 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786502156971023-7698-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 02:41:35 dist-test-slave-mn4r krb5kdc[32360](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 02:41:35 dist-test-slave-mn4r krb5kdc[32360](info): set up 2 sockets
Aug 12 02:41:35 dist-test-slave-mn4r krb5kdc[32360](info): commencing operation
krb5kdc: starting...
W20260812 02:41:36.248492 7698 mini_kdc.cc:121] Time spent starting KDC: real 1.191s user 0.004s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786502156971023-7698-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786502156971023-7698-0/krb5kdc/test-user.keytab.
Aug 12 02:41:36 dist-test-slave-mn4r krb5kdc[32360](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502496, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T02:41:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T02:41:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786502156971023-7698-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786502156971023-7698-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 02:41:36.577054 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786502156971023-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786502156971023-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786502156971023-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37041
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786502156971023-7698-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37041
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:41:36.924037 32375 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:41:36.924397 32375 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:41:36.924563 32375 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:41:36.931795 32375 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:41:36.931949 32375 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:41:36.932051 32375 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:41:36.932152 32375 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:41:36.932235 32375 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:41:36.942164 32375 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786502156971023-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786502156971023-7698-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37041
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786502156971023-7698-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37041
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786502156971023-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:41:36.943596 32375 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:41:36.945036 32375 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:41:36.957780 32381 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:41:36.957780 32382 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:41:36.957887 32384 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:41:36.958658 32375 server_base.cc:1034] running on GCE node
I20260812 02:41:36.959365 32375 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:41:36.960750 32375 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:41:36.961992 32375 hybrid_clock.cc:648] HybridClock initialized: now 1786502496961946 us; error 54 us; skew 500 ppm
I20260812 02:41:36.962375 32375 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:41:36 dist-test-slave-mn4r krb5kdc[32360](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502496, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:41:36.965428 32375 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 02:41:36.966913 32375 webserver.cc:460] Webserver started at http://127.0.0.1:34319/ using document root <none> and password file <none>
I20260812 02:41:36.967630 32375 fs_manager.cc:359] Metadata directory not provided
I20260812 02:41:36.967777 32375 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:41:36.968084 32375 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:41:36.971138 32375 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786502156971023-7698-0/minicluster-data/master-0/data/instance:
uuid: "e06af87bb2064b6caad2494f6ae7967f"
format_stamp: "Formatted at 2026-08-12 02:41:36 on dist-test-slave-mn4r"
server_key: "45e7e8cfb7d8a3923ba6207bf313724f"
server_key_iv: "baec42ea18f929bfabd0f3b070c6017c"
server_key_version: "encryptionkey@0"
I20260812 02:41:36.971897 32375 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786502156971023-7698-0/minicluster-data/master-0/wal/instance:
uuid: "e06af87bb2064b6caad2494f6ae7967f"
format_stamp: "Formatted at 2026-08-12 02:41:36 on dist-test-slave-mn4r"
server_key: "45e7e8cfb7d8a3923ba6207bf313724f"
server_key_iv: "baec42ea18f929bfabd0f3b070c6017c"
server_key_version: "encryptionkey@0"
I20260812 02:41:36.977092 32375 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260812 02:41:36.980742 32392 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:41:36.982220 32375 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 02:41:36.982404 32375 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786502156971023-7698-0/minicluster-data/master-0/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786502156971023-7698-0/minicluster-data/master-0/wal
uuid: "e06af87bb2064b6caad2494f6ae7967f"
format_stamp: "Formatted at 2026-08-12 02:41:36 on dist-test-slave-mn4r"
server_key: "45e7e8cfb7d8a3923ba6207bf313724f"
server_key_iv: "baec42ea18f929bfabd0f3b070c6017c"
server_key_version: "encryptionkey@0"
I20260812 02:41:36.982621 32375 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786502156971023-7698-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786502156971023-7698-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786502156971023-7698-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:41:37.009506 32375 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:41:37.013852 32375 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:41:37.014207 32375 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:41:37.033054 32375 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37041
I20260812 02:41:37.033126 32450 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37041 every 8 connection(s)
I20260812 02:41:37.034518 32375 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
I20260812 02:41:37.038373 32451 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:41:37.038700 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 32375
I20260812 02:41:37.038877 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786502156971023-7698-0/minicluster-data/master-0/wal/instance
I20260812 02:41:37.039474 7698 external_mini_cluster.cc:1428] Setting key 6fcdc2e59df289b8118c0a51d9395865
I20260812 02:41:37.051757 32451 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e06af87bb2064b6caad2494f6ae7967f: Bootstrap starting.
I20260812 02:41:37.052548 32454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:37.041412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59317 (local address 127.0.0.1:37041)
0812 02:41:37.042278 (+ 866us) server_negotiation.cc:206] Beginning negotiation
0812 02:41:37.042290 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 02:41:37.042331 (+ 41us) server_negotiation.cc:407] Connection header received
0812 02:41:37.043840 (+ 1509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:41:37.043866 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:41:37.044272 (+ 406us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:41:37.044679 (+ 407us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:41:37.045263 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:37.046615 (+ 1352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:37.047946 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:37.048671 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:37.048894 (+ 223us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:37.050234 (+ 1340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:41:37.050242 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:41:37.050256 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:41:37.050791 (+ 535us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:41:37.050866 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:41:37.050895 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:41:37.051063 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0812 02:41:37.051350 (+ 287us) server_negotiation.cc:299] Negotiation successful
0812 02:41:37.051518 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":580,"thread_start_us":344,"threads_started":1}
I20260812 02:41:37.055375 32451 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e06af87bb2064b6caad2494f6ae7967f: Neither blocks nor log segments found. Creating new log.
I20260812 02:41:37.056653 32451 log.cc:824] T 00000000000000000000000000000000 P e06af87bb2064b6caad2494f6ae7967f: Log is configured to *not* fsync() on all Append() calls
I20260812 02:41:37.056690 7698 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 02:41:37.059502 32451 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e06af87bb2064b6caad2494f6ae7967f: No bootstrap required, opened a new log
I20260812 02:41:37.068435 32454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:37.061318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48575 (local address 127.0.0.1:37041)
0812 02:41:37.061531 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0812 02:41:37.061537 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:41:37.061813 (+ 276us) server_negotiation.cc:407] Connection header received
0812 02:41:37.062014 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:41:37.062018 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:41:37.062170 (+ 152us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:41:37.062266 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:41:37.062900 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:37.063679 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:37.065659 (+ 1980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:37.066196 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:37.066299 (+ 103us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:37.067612 (+ 1313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:41:37.067617 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:41:37.067620 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:41:37.067638 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:41:37.067682 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:41:37.067699 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:41:37.067780 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 02:41:37.067999 (+ 219us) server_negotiation.cc:299] Negotiation successful
0812 02:41:37.068048 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260812 02:41:37.069751 32451 raft_consensus.cc:348] T 00000000000000000000000000000000 P e06af87bb2064b6caad2494f6ae7967f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e06af87bb2064b6caad2494f6ae7967f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37041 } }
I20260812 02:41:37.070223 32451 raft_consensus.cc:374] T 00000000000000000000000000000000 P e06af87bb2064b6caad2494f6ae7967f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:41:37.070336 32451 raft_consensus.cc:729] T 00000000000000000000000000000000 P e06af87bb2064b6caad2494f6ae7967f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e06af87bb2064b6caad2494f6ae7967f, State: Initialized, Role: FOLLOWER
I20260812 02:41:37.070945 32451 consensus_queue.cc:260] T 00000000000000000000000000000000 P e06af87bb2064b6caad2494f6ae7967f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e06af87bb2064b6caad2494f6ae7967f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37041 } }
I20260812 02:41:37.071215 32451 raft_consensus.cc:388] T 00000000000000000000000000000000 P e06af87bb2064b6caad2494f6ae7967f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 02:41:37.071376 32451 raft_consensus.cc:482] T 00000000000000000000000000000000 P e06af87bb2064b6caad2494f6ae7967f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 02:41:37.071565 32451 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e06af87bb2064b6caad2494f6ae7967f [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:41:37.073868 32451 raft_consensus.cc:504] T 00000000000000000000000000000000 P e06af87bb2064b6caad2494f6ae7967f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e06af87bb2064b6caad2494f6ae7967f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37041 } }
I20260812 02:41:37.074446 32451 leader_election.cc:302] T 00000000000000000000000000000000 P e06af87bb2064b6caad2494f6ae7967f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e06af87bb2064b6caad2494f6ae7967f; no voters:
I20260812 02:41:37.075079 32451 leader_election.cc:288] T 00000000000000000000000000000000 P e06af87bb2064b6caad2494f6ae7967f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 02:41:37.075243 32462 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e06af87bb2064b6caad2494f6ae7967f [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:41:37.076442 32462 raft_consensus.cc:686] T 00000000000000000000000000000000 P e06af87bb2064b6caad2494f6ae7967f [term 1 LEADER]: Becoming Leader. State: Replica: e06af87bb2064b6caad2494f6ae7967f, State: Running, Role: LEADER
I20260812 02:41:37.077149 32462 consensus_queue.cc:237] T 00000000000000000000000000000000 P e06af87bb2064b6caad2494f6ae7967f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e06af87bb2064b6caad2494f6ae7967f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37041 } }
I20260812 02:41:37.077462 32451 sys_catalog.cc:564] T 00000000000000000000000000000000 P e06af87bb2064b6caad2494f6ae7967f [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:41:37.080531 32464 sys_catalog.cc:455] T 00000000000000000000000000000000 P e06af87bb2064b6caad2494f6ae7967f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e06af87bb2064b6caad2494f6ae7967f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e06af87bb2064b6caad2494f6ae7967f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37041 } } }
I20260812 02:41:37.080498 32465 sys_catalog.cc:455] T 00000000000000000000000000000000 P e06af87bb2064b6caad2494f6ae7967f [sys.catalog]: SysCatalogTable state changed. Reason: New leader e06af87bb2064b6caad2494f6ae7967f. Latest consensus state: current_term: 1 leader_uuid: "e06af87bb2064b6caad2494f6ae7967f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e06af87bb2064b6caad2494f6ae7967f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37041 } } }
I20260812 02:41:37.081048 32464 sys_catalog.cc:458] T 00000000000000000000000000000000 P e06af87bb2064b6caad2494f6ae7967f [sys.catalog]: This master's current role is: LEADER
I20260812 02:41:37.081071 32465 sys_catalog.cc:458] T 00000000000000000000000000000000 P e06af87bb2064b6caad2494f6ae7967f [sys.catalog]: This master's current role is: LEADER
I20260812 02:41:37.086617 32477 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:41:37.091161 32477 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 02:41:37.099100 32477 catalog_manager.cc:1347] Generated new cluster ID: 26b44cf6375640e2b033a9d4cd638198
I20260812 02:41:37.099213 32477 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 02:41:37.121461 32477 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 02:41:37.122989 32477 catalog_manager.cc:1504] Loading token signing keys...
I20260812 02:41:37.135571 32477 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e06af87bb2064b6caad2494f6ae7967f: Generated new TSK 0
I20260812 02:41:37.136737 32477 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 02:41:37.148419 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 32375
W20260812 02:41:37.149725 32446 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:59317
W20260812 02:41:37.150539 7698 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 32375: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 02:41:37.150743 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 32375
W20260812 02:41:37.174734 32358 connection.cc:535] client connection to 127.0.0.1:37041 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37041): Connection reset by peer (error 104)
2026-08-12T02:41:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (2135 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786502156971023-7698-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 02:41:37 dist-test-slave-mn4r krb5kdc[32492](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 02:41:37 dist-test-slave-mn4r krb5kdc[32492](info): set up 2 sockets
Aug 12 02:41:37 dist-test-slave-mn4r krb5kdc[32492](info): commencing operation
krb5kdc: starting...
W20260812 02:41:38.392251 7698 mini_kdc.cc:121] Time spent starting KDC: real 1.199s user 0.000s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786502156971023-7698-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786502156971023-7698-0/krb5kdc/test-user.keytab.
Aug 12 02:41:38 dist-test-slave-mn4r krb5kdc[32492](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502498, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T02:41:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T02:41:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786502156971023-7698-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786502156971023-7698-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 02:41:38.717156 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786502156971023-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786502156971023-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786502156971023-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32899
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786502156971023-7698-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:32899
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:41:39.071429 32508 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:41:39.071719 32508 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:41:39.071854 32508 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:41:39.079155 32508 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:41:39.079311 32508 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:41:39.079417 32508 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:41:39.079527 32508 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:41:39.079605 32508 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:41:39.089831 32508 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786502156971023-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786502156971023-7698-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:32899
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786502156971023-7698-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32899
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786502156971023-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:41:39.091250 32508 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:41:39.092727 32508 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:41:39.104918 32515 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:41:39.104945 32518 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:41:39.105165 32516 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:41:39.105839 32508 server_base.cc:1034] running on GCE node
I20260812 02:41:39.106801 32508 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:41:39.108265 32508 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:41:39.109506 32508 hybrid_clock.cc:648] HybridClock initialized: now 1786502499109455 us; error 47 us; skew 500 ppm
I20260812 02:41:39.109928 32508 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:41:39 dist-test-slave-mn4r krb5kdc[32492](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502499, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:41:39.113229 32508 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 02:41:39.114820 32508 webserver.cc:460] Webserver started at http://127.0.0.1:42865/ using document root <none> and password file <none>
I20260812 02:41:39.115494 32508 fs_manager.cc:359] Metadata directory not provided
I20260812 02:41:39.115605 32508 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:41:39.115867 32508 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:41:39.118983 32508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786502156971023-7698-0/minicluster-data/master-0/data/instance:
uuid: "e7d0267d8e414ee6a4e3351baba32049"
format_stamp: "Formatted at 2026-08-12 02:41:39 on dist-test-slave-mn4r"
server_key: "25b20846e6f8e0db169948de11dd31c8"
server_key_iv: "960727e95283787e6aa3f825502bd209"
server_key_version: "encryptionkey@0"
I20260812 02:41:39.119740 32508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786502156971023-7698-0/minicluster-data/master-0/wal/instance:
uuid: "e7d0267d8e414ee6a4e3351baba32049"
format_stamp: "Formatted at 2026-08-12 02:41:39 on dist-test-slave-mn4r"
server_key: "25b20846e6f8e0db169948de11dd31c8"
server_key_iv: "960727e95283787e6aa3f825502bd209"
server_key_version: "encryptionkey@0"
I20260812 02:41:39.125082 32508 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.001s
I20260812 02:41:39.128904 32526 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:41:39.130432 32508 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 02:41:39.130633 32508 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786502156971023-7698-0/minicluster-data/master-0/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786502156971023-7698-0/minicluster-data/master-0/wal
uuid: "e7d0267d8e414ee6a4e3351baba32049"
format_stamp: "Formatted at 2026-08-12 02:41:39 on dist-test-slave-mn4r"
server_key: "25b20846e6f8e0db169948de11dd31c8"
server_key_iv: "960727e95283787e6aa3f825502bd209"
server_key_version: "encryptionkey@0"
I20260812 02:41:39.130877 32508 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786502156971023-7698-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786502156971023-7698-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786502156971023-7698-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:41:39.136759 32508 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:41:39.140826 32508 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:41:39.141122 32508 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:41:39.160271 32508 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:32899
I20260812 02:41:39.160342 32587 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:32899 every 8 connection(s)
I20260812 02:41:39.161734 32508 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
I20260812 02:41:39.165519 32588 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:41:39.170171 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 32508
I20260812 02:41:39.170353 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786502156971023-7698-0/minicluster-data/master-0/wal/instance
I20260812 02:41:39.170903 7698 external_mini_cluster.cc:1428] Setting key 0f98226cccd2caf13cb362f43bf71be2
I20260812 02:41:39.177145 32588 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e7d0267d8e414ee6a4e3351baba32049: Bootstrap starting.
I20260812 02:41:39.179435 32594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:39.172752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40515 (local address 127.0.0.1:32899)
0812 02:41:39.173551 (+ 799us) server_negotiation.cc:206] Beginning negotiation
0812 02:41:39.173563 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 02:41:39.173609 (+ 46us) server_negotiation.cc:407] Connection header received
0812 02:41:39.175924 (+ 2315us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:41:39.175949 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:41:39.176312 (+ 363us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:41:39.176628 (+ 316us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:41:39.177820 (+ 1192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:41:39.177826 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:41:39.177839 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:41:39.177853 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:41:39.177916 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:41:39.177936 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:41:39.178177 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0812 02:41:39.178288 (+ 111us) server_negotiation.cc:299] Negotiation successful
0812 02:41:39.178495 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":589,"thread_start_us":369,"threads_started":1}
I20260812 02:41:39.180672 32588 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e7d0267d8e414ee6a4e3351baba32049: Neither blocks nor log segments found. Creating new log.
I20260812 02:41:39.182055 32588 log.cc:824] T 00000000000000000000000000000000 P e7d0267d8e414ee6a4e3351baba32049: Log is configured to *not* fsync() on all Append() calls
I20260812 02:41:39.183351 7698 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 02:41:39.185286 32588 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e7d0267d8e414ee6a4e3351baba32049: No bootstrap required, opened a new log
I20260812 02:41:39.191395 32594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:39.188901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59463 (local address 127.0.0.1:32899)
0812 02:41:39.189050 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0812 02:41:39.189055 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:41:39.189376 (+ 321us) server_negotiation.cc:407] Connection header received
0812 02:41:39.189537 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:41:39.189545 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:41:39.189606 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:41:39.189690 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:41:39.190672 (+ 982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:41:39.190677 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:41:39.190680 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:41:39.190694 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:41:39.190736 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:41:39.190755 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:41:39.190847 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0812 02:41:39.190990 (+ 143us) server_negotiation.cc:299] Negotiation successful
0812 02:41:39.191041 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 02:41:39.195255 32588 raft_consensus.cc:348] T 00000000000000000000000000000000 P e7d0267d8e414ee6a4e3351baba32049 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7d0267d8e414ee6a4e3351baba32049" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32899 } }
I20260812 02:41:39.195657 32588 raft_consensus.cc:374] T 00000000000000000000000000000000 P e7d0267d8e414ee6a4e3351baba32049 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:41:39.195770 32588 raft_consensus.cc:729] T 00000000000000000000000000000000 P e7d0267d8e414ee6a4e3351baba32049 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7d0267d8e414ee6a4e3351baba32049, State: Initialized, Role: FOLLOWER
I20260812 02:41:39.196363 32588 consensus_queue.cc:260] T 00000000000000000000000000000000 P e7d0267d8e414ee6a4e3351baba32049 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7d0267d8e414ee6a4e3351baba32049" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32899 } }
I20260812 02:41:39.196614 32588 raft_consensus.cc:388] T 00000000000000000000000000000000 P e7d0267d8e414ee6a4e3351baba32049 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 02:41:39.196774 32588 raft_consensus.cc:482] T 00000000000000000000000000000000 P e7d0267d8e414ee6a4e3351baba32049 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 02:41:39.196949 32588 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e7d0267d8e414ee6a4e3351baba32049 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:41:39.199502 32588 raft_consensus.cc:504] T 00000000000000000000000000000000 P e7d0267d8e414ee6a4e3351baba32049 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7d0267d8e414ee6a4e3351baba32049" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32899 } }
I20260812 02:41:39.200027 32588 leader_election.cc:302] T 00000000000000000000000000000000 P e7d0267d8e414ee6a4e3351baba32049 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e7d0267d8e414ee6a4e3351baba32049; no voters:
I20260812 02:41:39.200659 32588 leader_election.cc:288] T 00000000000000000000000000000000 P e7d0267d8e414ee6a4e3351baba32049 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 02:41:39.200793 32602 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e7d0267d8e414ee6a4e3351baba32049 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:41:39.201571 32602 raft_consensus.cc:686] T 00000000000000000000000000000000 P e7d0267d8e414ee6a4e3351baba32049 [term 1 LEADER]: Becoming Leader. State: Replica: e7d0267d8e414ee6a4e3351baba32049, State: Running, Role: LEADER
I20260812 02:41:39.202212 32602 consensus_queue.cc:237] T 00000000000000000000000000000000 P e7d0267d8e414ee6a4e3351baba32049 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7d0267d8e414ee6a4e3351baba32049" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32899 } }
I20260812 02:41:39.202991 32588 sys_catalog.cc:564] T 00000000000000000000000000000000 P e7d0267d8e414ee6a4e3351baba32049 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:41:39.205279 32603 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7d0267d8e414ee6a4e3351baba32049 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e7d0267d8e414ee6a4e3351baba32049" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7d0267d8e414ee6a4e3351baba32049" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32899 } } }
I20260812 02:41:39.205251 32604 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7d0267d8e414ee6a4e3351baba32049 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e7d0267d8e414ee6a4e3351baba32049. Latest consensus state: current_term: 1 leader_uuid: "e7d0267d8e414ee6a4e3351baba32049" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7d0267d8e414ee6a4e3351baba32049" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32899 } } }
I20260812 02:41:39.205787 32603 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7d0267d8e414ee6a4e3351baba32049 [sys.catalog]: This master's current role is: LEADER
I20260812 02:41:39.205865 32604 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7d0267d8e414ee6a4e3351baba32049 [sys.catalog]: This master's current role is: LEADER
I20260812 02:41:39.209811 32610 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:41:39.215492 32610 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 02:41:39.224852 32610 catalog_manager.cc:1347] Generated new cluster ID: 41fc0b9c398d4c7888c2f6214bfbc67c
I20260812 02:41:39.224978 32610 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 02:41:39.241706 32610 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 02:41:39.242199 32610 catalog_manager.cc:1504] Loading token signing keys...
I20260812 02:41:39.254606 32610 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e7d0267d8e414ee6a4e3351baba32049: Generated new TSK 0
I20260812 02:41:39.255666 32610 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 02:41:39.265105 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 32508
W20260812 02:41:39.266323 32582 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:40515
W20260812 02:41:39.267043 7698 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 32508: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 02:41:39.267189 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 32508
2026-08-12T02:41:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (2116 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786502156971023-7698-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 02:41:39 dist-test-slave-mn4r krb5kdc[32628](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 02:41:39 dist-test-slave-mn4r krb5kdc[32628](info): set up 2 sockets
Aug 12 02:41:39 dist-test-slave-mn4r krb5kdc[32628](info): commencing operation
krb5kdc: starting...
W20260812 02:41:40.568387 7698 mini_kdc.cc:121] Time spent starting KDC: real 1.259s user 0.005s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786502156971023-7698-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786502156971023-7698-0/krb5kdc/test-user.keytab.
Aug 12 02:41:40 dist-test-slave-mn4r krb5kdc[32628](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502500, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T02:41:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T02:41:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786502156971023-7698-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786502156971023-7698-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 02:41:40.884883 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786502156971023-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786502156971023-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786502156971023-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42753
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786502156971023-7698-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42753
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:41:41.231236 32643 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:41:41.231518 32643 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:41:41.231643 32643 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:41:41.238751 32643 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:41:41.238893 32643 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:41:41.238991 32643 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:41:41.239105 32643 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:41:41.239173 32643 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:41:41.248813 32643 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786502156971023-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786502156971023-7698-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42753
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786502156971023-7698-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42753
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786502156971023-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:41:41.250191 32643 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:41:41.251578 32643 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:41:41.263875 32650 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:41:41.263782 32653 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:41:41.263839 32651 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:41:41.264912 32643 server_base.cc:1034] running on GCE node
I20260812 02:41:41.265753 32643 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:41:41.267328 32643 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:41:41.268584 32643 hybrid_clock.cc:648] HybridClock initialized: now 1786502501268534 us; error 64 us; skew 500 ppm
I20260812 02:41:41.268988 32643 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:41:41 dist-test-slave-mn4r krb5kdc[32628](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502501, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:41:41.275897 32643 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 02:41:41.277511 32643 webserver.cc:460] Webserver started at http://127.0.0.1:34121/ using document root <none> and password file <none>
I20260812 02:41:41.278311 32643 fs_manager.cc:359] Metadata directory not provided
I20260812 02:41:41.278456 32643 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:41:41.278775 32643 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:41:41.281993 32643 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786502156971023-7698-0/minicluster-data/master-0/data/instance:
uuid: "3a87b470fe784e6c92f0706d5033205c"
format_stamp: "Formatted at 2026-08-12 02:41:41 on dist-test-slave-mn4r"
server_key: "f5b392677e848d629f980948185800b8"
server_key_iv: "b0bd47ee96ac613c2912ace143a97bcd"
server_key_version: "encryptionkey@0"
I20260812 02:41:41.282902 32643 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786502156971023-7698-0/minicluster-data/master-0/wal/instance:
uuid: "3a87b470fe784e6c92f0706d5033205c"
format_stamp: "Formatted at 2026-08-12 02:41:41 on dist-test-slave-mn4r"
server_key: "f5b392677e848d629f980948185800b8"
server_key_iv: "b0bd47ee96ac613c2912ace143a97bcd"
server_key_version: "encryptionkey@0"
I20260812 02:41:41.288435 32643 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 02:41:41.292168 32661 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:41:41.293494 32643 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 02:41:41.293681 32643 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786502156971023-7698-0/minicluster-data/master-0/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786502156971023-7698-0/minicluster-data/master-0/wal
uuid: "3a87b470fe784e6c92f0706d5033205c"
format_stamp: "Formatted at 2026-08-12 02:41:41 on dist-test-slave-mn4r"
server_key: "f5b392677e848d629f980948185800b8"
server_key_iv: "b0bd47ee96ac613c2912ace143a97bcd"
server_key_version: "encryptionkey@0"
I20260812 02:41:41.293885 32643 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786502156971023-7698-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786502156971023-7698-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786502156971023-7698-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:41:41.316811 32643 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:41:41.320984 32643 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:41:41.321290 32643 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:41:41.340762 32643 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42753
I20260812 02:41:41.340809 32722 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42753 every 8 connection(s)
I20260812 02:41:41.342366 32643 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
I20260812 02:41:41.346546 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 32643
I20260812 02:41:41.346737 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786502156971023-7698-0/minicluster-data/master-0/wal/instance
I20260812 02:41:41.347342 7698 external_mini_cluster.cc:1428] Setting key df99b84d54aea748b5b2236232722a92
I20260812 02:41:41.347491 32723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:41:41.363214 32723 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a87b470fe784e6c92f0706d5033205c: Bootstrap starting.
Aug 12 02:41:41 dist-test-slave-mn4r krb5kdc[32628](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502500, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 02:41:41.367480 32723 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3a87b470fe784e6c92f0706d5033205c: Neither blocks nor log segments found. Creating new log.
I20260812 02:41:41.368748 32723 log.cc:824] T 00000000000000000000000000000000 P 3a87b470fe784e6c92f0706d5033205c: Log is configured to *not* fsync() on all Append() calls
I20260812 02:41:41.372532 32723 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a87b470fe784e6c92f0706d5033205c: No bootstrap required, opened a new log
I20260812 02:41:41.373304 32727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:41.350054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44193 (local address 127.0.0.1:42753)
0812 02:41:41.350946 (+ 892us) server_negotiation.cc:206] Beginning negotiation
0812 02:41:41.350967 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0812 02:41:41.351020 (+ 53us) server_negotiation.cc:407] Connection header received
0812 02:41:41.353094 (+ 2074us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:41:41.353124 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:41:41.353709 (+ 585us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:41:41.354214 (+ 505us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:41:41.355567 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:41.357588 (+ 2021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:41.358970 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:41.360151 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:41.360362 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:41.364941 (+ 4579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:41:41.364950 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:41:41.364970 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:41:41.365010 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:41:41.368419 (+ 3409us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:41:41.369479 (+ 1060us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:41:41.369492 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:41:41.369503 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:41:41.369775 (+ 272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:41:41.370292 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:41:41.370301 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:41:41.370308 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:41:41.370936 (+ 628us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:41:41.371180 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0812 02:41:41.371707 (+ 527us) server_negotiation.cc:299] Negotiation successful
0812 02:41:41.372107 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":603,"thread_start_us":327,"threads_started":1}
I20260812 02:41:41.377287 7698 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 02:41:41.382603 32723 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3a87b470fe784e6c92f0706d5033205c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a87b470fe784e6c92f0706d5033205c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42753 } }
I20260812 02:41:41.383144 32723 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3a87b470fe784e6c92f0706d5033205c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:41:41.383266 32723 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3a87b470fe784e6c92f0706d5033205c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a87b470fe784e6c92f0706d5033205c, State: Initialized, Role: FOLLOWER
I20260812 02:41:41.383972 32723 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3a87b470fe784e6c92f0706d5033205c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a87b470fe784e6c92f0706d5033205c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42753 } }
I20260812 02:41:41.384315 32723 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3a87b470fe784e6c92f0706d5033205c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 02:41:41.384473 32723 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3a87b470fe784e6c92f0706d5033205c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 02:41:41.384668 32723 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3a87b470fe784e6c92f0706d5033205c [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:41:41.387859 32723 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3a87b470fe784e6c92f0706d5033205c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a87b470fe784e6c92f0706d5033205c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42753 } }
I20260812 02:41:41.388495 32723 leader_election.cc:302] T 00000000000000000000000000000000 P 3a87b470fe784e6c92f0706d5033205c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3a87b470fe784e6c92f0706d5033205c; no voters:
I20260812 02:41:41.389226 32723 leader_election.cc:288] T 00000000000000000000000000000000 P 3a87b470fe784e6c92f0706d5033205c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 02:41:41.389322 32735 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3a87b470fe784e6c92f0706d5033205c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:41:41.392005 32723 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3a87b470fe784e6c92f0706d5033205c [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:41:41.392580 32735 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3a87b470fe784e6c92f0706d5033205c [term 1 LEADER]: Becoming Leader. State: Replica: 3a87b470fe784e6c92f0706d5033205c, State: Running, Role: LEADER
I20260812 02:41:41.393266 32735 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3a87b470fe784e6c92f0706d5033205c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a87b470fe784e6c92f0706d5033205c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42753 } }
I20260812 02:41:41.396549 32737 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a87b470fe784e6c92f0706d5033205c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3a87b470fe784e6c92f0706d5033205c. Latest consensus state: current_term: 1 leader_uuid: "3a87b470fe784e6c92f0706d5033205c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a87b470fe784e6c92f0706d5033205c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42753 } } }
I20260812 02:41:41.397001 32737 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a87b470fe784e6c92f0706d5033205c [sys.catalog]: This master's current role is: LEADER
I20260812 02:41:41.396960 32736 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a87b470fe784e6c92f0706d5033205c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3a87b470fe784e6c92f0706d5033205c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a87b470fe784e6c92f0706d5033205c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42753 } } }
I20260812 02:41:41.397275 32736 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a87b470fe784e6c92f0706d5033205c [sys.catalog]: This master's current role is: LEADER
I20260812 02:41:41.402532 32727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:41.382685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42253 (local address 127.0.0.1:42753)
0812 02:41:41.382906 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0812 02:41:41.382915 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 02:41:41.383440 (+ 525us) server_negotiation.cc:407] Connection header received
0812 02:41:41.383661 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:41:41.383670 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:41:41.383753 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:41:41.383833 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:41:41.385883 (+ 2050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:41.386709 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:41.388453 (+ 1744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:41.388973 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:41.389124 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:41.392192 (+ 3068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:41:41.392200 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:41:41.392207 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:41:41.392252 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:41:41.396551 (+ 4299us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:41:41.398005 (+ 1454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:41:41.398011 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:41:41.398015 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:41:41.398209 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:41:41.398923 (+ 714us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:41:41.398931 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:41:41.398937 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:41:41.399505 (+ 568us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:41:41.399851 (+ 346us) server_negotiation.cc:1035] Waiting for connection context
0812 02:41:41.401654 (+ 1803us) server_negotiation.cc:299] Negotiation successful
0812 02:41:41.401986 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 02:41:41.411031 32745 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:41:41.415617 32745 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 02:41:41.425624 32745 catalog_manager.cc:1347] Generated new cluster ID: 3c08615ccd3d491694a2ba616b62be4d
I20260812 02:41:41.425769 32745 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 02:41:41.444834 32745 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 02:41:41.446926 32745 catalog_manager.cc:1504] Loading token signing keys...
I20260812 02:41:41.459556 32745 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3a87b470fe784e6c92f0706d5033205c: Generated new TSK 0
I20260812 02:41:41.460436 32745 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 02:41:41.475775 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 32643
I20260812 02:41:41.476862 32716 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:41:41.569861 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 32643
2026-08-12T02:41:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (2303 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786502156971023-7698-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 02:41:41 dist-test-slave-mn4r krb5kdc[32761](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 02:41:41 dist-test-slave-mn4r krb5kdc[32761](info): set up 2 sockets
Aug 12 02:41:41 dist-test-slave-mn4r krb5kdc[32761](info): commencing operation
krb5kdc: starting...
W20260812 02:41:42.848105 7698 mini_kdc.cc:121] Time spent starting KDC: real 1.234s user 0.005s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786502156971023-7698-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786502156971023-7698-0/krb5kdc/test-user.keytab.
Aug 12 02:41:43 dist-test-slave-mn4r krb5kdc[32761](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502503, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T02:41:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T02:41:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786502156971023-7698-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786502156971023-7698-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 02:41:43.178855 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786502156971023-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786502156971023-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786502156971023-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38877
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786502156971023-7698-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38877
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:41:43.540745 311 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:41:43.541119 311 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:41:43.541217 311 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:41:43.548700 311 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:41:43.548832 311 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:41:43.548880 311 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:41:43.548957 311 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:41:43.549003 311 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:41:43.559697 311 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786502156971023-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786502156971023-7698-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38877
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786502156971023-7698-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38877
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786502156971023-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:41:43.561062 311 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:41:43.562515 311 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:41:43.574793 317 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:41:43.574826 320 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:41:43.574867 318 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:41:43.575448 311 server_base.cc:1034] running on GCE node
I20260812 02:41:43.576308 311 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:41:43.577833 311 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:41:43.579087 311 hybrid_clock.cc:648] HybridClock initialized: now 1786502503579020 us; error 46 us; skew 500 ppm
I20260812 02:41:43.579454 311 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:41:43 dist-test-slave-mn4r krb5kdc[32761](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502503, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:41:43.582805 311 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 02:41:43.584353 311 webserver.cc:460] Webserver started at http://127.0.0.1:34477/ using document root <none> and password file <none>
I20260812 02:41:43.585139 311 fs_manager.cc:359] Metadata directory not provided
I20260812 02:41:43.585273 311 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:41:43.585606 311 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:41:43.588819 311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786502156971023-7698-0/minicluster-data/master-0/data/instance:
uuid: "85c57d3308934059aa820ad28a8f7c88"
format_stamp: "Formatted at 2026-08-12 02:41:43 on dist-test-slave-mn4r"
server_key: "0e39d2f2b1dc1619cde29942d02b89be"
server_key_iv: "012bf5e09f1f9987ae353d0bb7cabea4"
server_key_version: "encryptionkey@0"
I20260812 02:41:43.589622 311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786502156971023-7698-0/minicluster-data/master-0/wal/instance:
uuid: "85c57d3308934059aa820ad28a8f7c88"
format_stamp: "Formatted at 2026-08-12 02:41:43 on dist-test-slave-mn4r"
server_key: "0e39d2f2b1dc1619cde29942d02b89be"
server_key_iv: "012bf5e09f1f9987ae353d0bb7cabea4"
server_key_version: "encryptionkey@0"
I20260812 02:41:43.595152 311 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260812 02:41:43.598747 328 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:41:43.600082 311 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 02:41:43.600266 311 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786502156971023-7698-0/minicluster-data/master-0/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786502156971023-7698-0/minicluster-data/master-0/wal
uuid: "85c57d3308934059aa820ad28a8f7c88"
format_stamp: "Formatted at 2026-08-12 02:41:43 on dist-test-slave-mn4r"
server_key: "0e39d2f2b1dc1619cde29942d02b89be"
server_key_iv: "012bf5e09f1f9987ae353d0bb7cabea4"
server_key_version: "encryptionkey@0"
I20260812 02:41:43.600469 311 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786502156971023-7698-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786502156971023-7698-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786502156971023-7698-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:41:43.612628 311 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:41:43.616752 311 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:41:43.617055 311 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:41:43.637415 311 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38877
I20260812 02:41:43.637436 389 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38877 every 8 connection(s)
I20260812 02:41:43.638978 311 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
I20260812 02:41:43.641405 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 311
I20260812 02:41:43.641587 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786502156971023-7698-0/minicluster-data/master-0/wal/instance
I20260812 02:41:43.642053 7698 external_mini_cluster.cc:1428] Setting key 2413f8d89bf63c33e7c8b368fa01a394
I20260812 02:41:43.643375 390 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:41:43.657668 394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:43.644024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57739 (local address 127.0.0.1:38877)
0812 02:41:43.644858 (+ 834us) server_negotiation.cc:206] Beginning negotiation
0812 02:41:43.644874 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 02:41:43.644929 (+ 55us) server_negotiation.cc:407] Connection header received
0812 02:41:43.646960 (+ 2031us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:41:43.646993 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:41:43.647601 (+ 608us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:41:43.648108 (+ 507us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:41:43.648679 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:43.650853 (+ 2174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:43.652204 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:43.653262 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:43.653452 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:43.654623 (+ 1171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:41:43.654634 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:41:43.654653 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:41:43.655469 (+ 816us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:41:43.655582 (+ 113us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:41:43.655614 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:41:43.655744 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0812 02:41:43.655964 (+ 220us) server_negotiation.cc:299] Negotiation successful
0812 02:41:43.656272 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":586,"thread_start_us":333,"threads_started":1}
I20260812 02:41:43.659056 390 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 85c57d3308934059aa820ad28a8f7c88: Bootstrap starting.
I20260812 02:41:43.661540 7698 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 02:41:43.662678 390 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 85c57d3308934059aa820ad28a8f7c88: Neither blocks nor log segments found. Creating new log.
I20260812 02:41:43.663949 390 log.cc:824] T 00000000000000000000000000000000 P 85c57d3308934059aa820ad28a8f7c88: Log is configured to *not* fsync() on all Append() calls
I20260812 02:41:43.666961 390 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 85c57d3308934059aa820ad28a8f7c88: No bootstrap required, opened a new log
I20260812 02:41:43.674046 394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:43.667224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60243 (local address 127.0.0.1:38877)
0812 02:41:43.667464 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0812 02:41:43.667475 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 02:41:43.667938 (+ 463us) server_negotiation.cc:407] Connection header received
0812 02:41:43.668082 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:41:43.668088 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:41:43.668167 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:41:43.668290 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:41:43.668798 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:43.669926 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:43.671292 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:43.672025 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:43.672174 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:43.673245 (+ 1071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:41:43.673252 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:41:43.673257 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:41:43.673285 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:41:43.673331 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:41:43.673351 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:41:43.673444 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0812 02:41:43.673573 (+ 129us) server_negotiation.cc:299] Negotiation successful
0812 02:41:43.673673 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 02:41:43.676641 390 raft_consensus.cc:348] T 00000000000000000000000000000000 P 85c57d3308934059aa820ad28a8f7c88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85c57d3308934059aa820ad28a8f7c88" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38877 } }
I20260812 02:41:43.677060 390 raft_consensus.cc:374] T 00000000000000000000000000000000 P 85c57d3308934059aa820ad28a8f7c88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:41:43.677181 390 raft_consensus.cc:729] T 00000000000000000000000000000000 P 85c57d3308934059aa820ad28a8f7c88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85c57d3308934059aa820ad28a8f7c88, State: Initialized, Role: FOLLOWER
I20260812 02:41:43.677927 390 consensus_queue.cc:260] T 00000000000000000000000000000000 P 85c57d3308934059aa820ad28a8f7c88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85c57d3308934059aa820ad28a8f7c88" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38877 } }
I20260812 02:41:43.678217 390 raft_consensus.cc:388] T 00000000000000000000000000000000 P 85c57d3308934059aa820ad28a8f7c88 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 02:41:43.678328 390 raft_consensus.cc:482] T 00000000000000000000000000000000 P 85c57d3308934059aa820ad28a8f7c88 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 02:41:43.678473 390 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 85c57d3308934059aa820ad28a8f7c88 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:41:43.680680 390 raft_consensus.cc:504] T 00000000000000000000000000000000 P 85c57d3308934059aa820ad28a8f7c88 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85c57d3308934059aa820ad28a8f7c88" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38877 } }
I20260812 02:41:43.681228 390 leader_election.cc:302] T 00000000000000000000000000000000 P 85c57d3308934059aa820ad28a8f7c88 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 85c57d3308934059aa820ad28a8f7c88; no voters:
I20260812 02:41:43.682070 390 leader_election.cc:288] T 00000000000000000000000000000000 P 85c57d3308934059aa820ad28a8f7c88 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 02:41:43.682282 404 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 85c57d3308934059aa820ad28a8f7c88 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:41:43.683024 404 raft_consensus.cc:686] T 00000000000000000000000000000000 P 85c57d3308934059aa820ad28a8f7c88 [term 1 LEADER]: Becoming Leader. State: Replica: 85c57d3308934059aa820ad28a8f7c88, State: Running, Role: LEADER
I20260812 02:41:43.683573 404 consensus_queue.cc:237] T 00000000000000000000000000000000 P 85c57d3308934059aa820ad28a8f7c88 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85c57d3308934059aa820ad28a8f7c88" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38877 } }
I20260812 02:41:43.684317 390 sys_catalog.cc:564] T 00000000000000000000000000000000 P 85c57d3308934059aa820ad28a8f7c88 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:41:43.686801 406 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85c57d3308934059aa820ad28a8f7c88 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 85c57d3308934059aa820ad28a8f7c88. Latest consensus state: current_term: 1 leader_uuid: "85c57d3308934059aa820ad28a8f7c88" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85c57d3308934059aa820ad28a8f7c88" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38877 } } }
I20260812 02:41:43.686770 405 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85c57d3308934059aa820ad28a8f7c88 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "85c57d3308934059aa820ad28a8f7c88" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85c57d3308934059aa820ad28a8f7c88" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38877 } } }
I20260812 02:41:43.687232 406 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85c57d3308934059aa820ad28a8f7c88 [sys.catalog]: This master's current role is: LEADER
I20260812 02:41:43.687288 405 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85c57d3308934059aa820ad28a8f7c88 [sys.catalog]: This master's current role is: LEADER
I20260812 02:41:43.690171 414 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:41:43.695727 414 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 02:41:43.705206 414 catalog_manager.cc:1347] Generated new cluster ID: 4909018e409d4aaf85365ae08ae13497
I20260812 02:41:43.705324 414 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 02:41:43.718900 414 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 02:41:43.720311 414 catalog_manager.cc:1504] Loading token signing keys...
I20260812 02:41:43.726776 414 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 85c57d3308934059aa820ad28a8f7c88: Generated new TSK 0
I20260812 02:41:43.727648 414 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 02:41:43.743880 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 311
W20260812 02:41:43.745261 385 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:57739
W20260812 02:41:43.746133 7698 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 311: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 02:41:43.746294 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 311
2026-08-12T02:41:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (2176 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786502156971023-7698-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 02:41:43 dist-test-slave-mn4r krb5kdc[431](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 02:41:43 dist-test-slave-mn4r krb5kdc[431](info): set up 2 sockets
Aug 12 02:41:43 dist-test-slave-mn4r krb5kdc[431](info): commencing operation
krb5kdc: starting...
W20260812 02:41:44.976111 7698 mini_kdc.cc:121] Time spent starting KDC: real 1.186s user 0.012s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786502156971023-7698-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786502156971023-7698-0/krb5kdc/test-user.keytab.
Aug 12 02:41:45 dist-test-slave-mn4r krb5kdc[431](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502505, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T02:41:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T02:41:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786502156971023-7698-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786502156971023-7698-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 02:41:45.298450 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786502156971023-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786502156971023-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786502156971023-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42079
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786502156971023-7698-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42079
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:41:45.651394 446 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:41:45.651686 446 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:41:45.651863 446 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:41:45.659272 446 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:41:45.659417 446 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:41:45.659504 446 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:41:45.659605 446 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:41:45.659680 446 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:41:45.669983 446 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786502156971023-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786502156971023-7698-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42079
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786502156971023-7698-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42079
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786502156971023-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:41:45.671358 446 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:41:45.672709 446 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:41:45.684854 451 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:41:45.684834 452 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:41:45.685324 454 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:41:45.685578 446 server_base.cc:1034] running on GCE node
I20260812 02:41:45.686508 446 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:41:45.687960 446 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:41:45.689205 446 hybrid_clock.cc:648] HybridClock initialized: now 1786502505689139 us; error 58 us; skew 500 ppm
I20260812 02:41:45.689581 446 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:41:45 dist-test-slave-mn4r krb5kdc[431](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502505, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:41:45.692770 446 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 02:41:45.694259 446 webserver.cc:460] Webserver started at http://127.0.0.1:42307/ using document root <none> and password file <none>
I20260812 02:41:45.694929 446 fs_manager.cc:359] Metadata directory not provided
I20260812 02:41:45.695043 446 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:41:45.695295 446 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:41:45.698375 446 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786502156971023-7698-0/minicluster-data/master-0/data/instance:
uuid: "6c99759c9fd74cec9073a9bc9df1f18a"
format_stamp: "Formatted at 2026-08-12 02:41:45 on dist-test-slave-mn4r"
server_key: "49914e182711c538b77fa6ce4e1ec574"
server_key_iv: "8a15573c403b31f0d9941efd7dd4da45"
server_key_version: "encryptionkey@0"
I20260812 02:41:45.699330 446 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786502156971023-7698-0/minicluster-data/master-0/wal/instance:
uuid: "6c99759c9fd74cec9073a9bc9df1f18a"
format_stamp: "Formatted at 2026-08-12 02:41:45 on dist-test-slave-mn4r"
server_key: "49914e182711c538b77fa6ce4e1ec574"
server_key_iv: "8a15573c403b31f0d9941efd7dd4da45"
server_key_version: "encryptionkey@0"
I20260812 02:41:45.704546 446 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 02:41:45.707962 463 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:41:45.709344 446 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 02:41:45.709535 446 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786502156971023-7698-0/minicluster-data/master-0/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786502156971023-7698-0/minicluster-data/master-0/wal
uuid: "6c99759c9fd74cec9073a9bc9df1f18a"
format_stamp: "Formatted at 2026-08-12 02:41:45 on dist-test-slave-mn4r"
server_key: "49914e182711c538b77fa6ce4e1ec574"
server_key_iv: "8a15573c403b31f0d9941efd7dd4da45"
server_key_version: "encryptionkey@0"
I20260812 02:41:45.709736 446 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786502156971023-7698-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786502156971023-7698-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786502156971023-7698-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:41:45.715430 446 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:41:45.719467 446 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:41:45.719764 446 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:41:45.738137 446 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42079
I20260812 02:41:45.738190 529 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42079 every 8 connection(s)
I20260812 02:41:45.739627 446 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
I20260812 02:41:45.743377 530 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:41:45.749688 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 446
I20260812 02:41:45.749989 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786502156971023-7698-0/minicluster-data/master-0/wal/instance
I20260812 02:41:45.750494 7698 external_mini_cluster.cc:1428] Setting key 63bb64320d3bef129d558ce46434ef5e
I20260812 02:41:45.754280 530 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c99759c9fd74cec9073a9bc9df1f18a: Bootstrap starting.
I20260812 02:41:45.757665 530 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6c99759c9fd74cec9073a9bc9df1f18a: Neither blocks nor log segments found. Creating new log.
I20260812 02:41:45.758440 533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:45.752276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34343 (local address 127.0.0.1:42079)
0812 02:41:45.753073 (+ 797us) server_negotiation.cc:206] Beginning negotiation
0812 02:41:45.753086 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 02:41:45.753125 (+ 39us) server_negotiation.cc:407] Connection header received
0812 02:41:45.754962 (+ 1837us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:41:45.754984 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:41:45.755310 (+ 326us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:41:45.755722 (+ 412us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:41:45.756748 (+ 1026us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:41:45.756753 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:41:45.756766 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:41:45.756783 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:41:45.756840 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:41:45.756859 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:41:45.757050 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0812 02:41:45.757190 (+ 140us) server_negotiation.cc:299] Negotiation successful
0812 02:41:45.757381 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":574,"thread_start_us":349,"threads_started":1}
I20260812 02:41:45.759006 530 log.cc:824] T 00000000000000000000000000000000 P 6c99759c9fd74cec9073a9bc9df1f18a: Log is configured to *not* fsync() on all Append() calls
I20260812 02:41:45.762104 530 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c99759c9fd74cec9073a9bc9df1f18a: No bootstrap required, opened a new log
I20260812 02:41:45.762240 7698 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 02:41:45.769035 530 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6c99759c9fd74cec9073a9bc9df1f18a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c99759c9fd74cec9073a9bc9df1f18a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42079 } }
I20260812 02:41:45.769381 530 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6c99759c9fd74cec9073a9bc9df1f18a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:41:45.769510 530 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6c99759c9fd74cec9073a9bc9df1f18a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c99759c9fd74cec9073a9bc9df1f18a, State: Initialized, Role: FOLLOWER
I20260812 02:41:45.770183 530 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6c99759c9fd74cec9073a9bc9df1f18a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c99759c9fd74cec9073a9bc9df1f18a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42079 } }
I20260812 02:41:45.770429 530 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6c99759c9fd74cec9073a9bc9df1f18a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 02:41:45.770581 530 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6c99759c9fd74cec9073a9bc9df1f18a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 02:41:45.770758 530 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6c99759c9fd74cec9073a9bc9df1f18a [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:41:45.773480 530 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6c99759c9fd74cec9073a9bc9df1f18a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c99759c9fd74cec9073a9bc9df1f18a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42079 } }
I20260812 02:41:45.773927 533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:45.771418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34433 (local address 127.0.0.1:42079)
0812 02:41:45.771613 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0812 02:41:45.771619 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:41:45.772015 (+ 396us) server_negotiation.cc:407] Connection header received
0812 02:41:45.772153 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:41:45.772159 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:41:45.772225 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:41:45.772329 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:41:45.773278 (+ 949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:41:45.773285 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:41:45.773293 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:41:45.773313 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:41:45.773364 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:41:45.773386 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:41:45.773537 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0812 02:41:45.773585 (+ 48us) server_negotiation.cc:299] Negotiation successful
0812 02:41:45.773644 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 02:41:45.774119 530 leader_election.cc:302] T 00000000000000000000000000000000 P 6c99759c9fd74cec9073a9bc9df1f18a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6c99759c9fd74cec9073a9bc9df1f18a; no voters:
I20260812 02:41:45.774811 530 leader_election.cc:288] T 00000000000000000000000000000000 P 6c99759c9fd74cec9073a9bc9df1f18a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 02:41:45.774923 541 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6c99759c9fd74cec9073a9bc9df1f18a [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:41:45.776075 541 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6c99759c9fd74cec9073a9bc9df1f18a [term 1 LEADER]: Becoming Leader. State: Replica: 6c99759c9fd74cec9073a9bc9df1f18a, State: Running, Role: LEADER
I20260812 02:41:45.776698 541 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6c99759c9fd74cec9073a9bc9df1f18a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c99759c9fd74cec9073a9bc9df1f18a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42079 } }
I20260812 02:41:45.777313 530 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6c99759c9fd74cec9073a9bc9df1f18a [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:41:45.780174 544 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c99759c9fd74cec9073a9bc9df1f18a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6c99759c9fd74cec9073a9bc9df1f18a. Latest consensus state: current_term: 1 leader_uuid: "6c99759c9fd74cec9073a9bc9df1f18a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c99759c9fd74cec9073a9bc9df1f18a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42079 } } }
I20260812 02:41:45.781049 544 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c99759c9fd74cec9073a9bc9df1f18a [sys.catalog]: This master's current role is: LEADER
I20260812 02:41:45.780213 543 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c99759c9fd74cec9073a9bc9df1f18a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6c99759c9fd74cec9073a9bc9df1f18a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c99759c9fd74cec9073a9bc9df1f18a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42079 } } }
I20260812 02:41:45.781431 543 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c99759c9fd74cec9073a9bc9df1f18a [sys.catalog]: This master's current role is: LEADER
I20260812 02:41:45.787546 549 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:41:45.792656 549 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 02:41:45.801800 549 catalog_manager.cc:1347] Generated new cluster ID: 89e29bf9d5ef41b1adb01f479a7a92e2
I20260812 02:41:45.801940 549 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 02:41:45.822346 549 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 02:41:45.822752 549 catalog_manager.cc:1504] Loading token signing keys...
I20260812 02:41:45.828975 549 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6c99759c9fd74cec9073a9bc9df1f18a: Generated new TSK 0
I20260812 02:41:45.829713 549 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 02:41:45.840622 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 446
W20260812 02:41:45.841830 524 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:34343
W20260812 02:41:45.842638 7698 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 446: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 02:41:45.842783 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 446
2026-08-12T02:41:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (2094 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786502156971023-7698-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 02:41:45 dist-test-slave-mn4r krb5kdc[569](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 02:41:45 dist-test-slave-mn4r krb5kdc[569](info): set up 2 sockets
Aug 12 02:41:45 dist-test-slave-mn4r krb5kdc[569](info): commencing operation
krb5kdc: starting...
W20260812 02:41:47.096664 7698 mini_kdc.cc:121] Time spent starting KDC: real 1.211s user 0.000s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786502156971023-7698-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786502156971023-7698-0/krb5kdc/test-user.keytab.
Aug 12 02:41:47 dist-test-slave-mn4r krb5kdc[569](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502507, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T02:41:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T02:41:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786502156971023-7698-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786502156971023-7698-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 02:41:47.427253 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786502156971023-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786502156971023-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786502156971023-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44095
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786502156971023-7698-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33025
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44095
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:41:47.782377 582 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:41:47.782701 582 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:41:47.782871 582 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:41:47.790467 582 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:41:47.790596 582 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:41:47.790709 582 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:41:47.790822 582 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:41:47.790876 582 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:41:47.801414 582 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33025
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786502156971023-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786502156971023-7698-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44095
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786502156971023-7698-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44095
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786502156971023-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:41:47.802974 582 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:41:47.804459 582 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:41:47.817127 591 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:41:47.817024 589 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:41:47.817099 588 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:41:47.817888 582 server_base.cc:1034] running on GCE node
I20260812 02:41:47.818780 582 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:41:47.820248 582 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:41:47.821483 582 hybrid_clock.cc:648] HybridClock initialized: now 1786502507821423 us; error 54 us; skew 500 ppm
I20260812 02:41:47.821862 582 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:41:47 dist-test-slave-mn4r krb5kdc[569](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502507, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:41:47.830554 582 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 02:41:47.832288 582 webserver.cc:460] Webserver started at http://127.0.0.1:34589/ using document root <none> and password file <none>
I20260812 02:41:47.833083 582 fs_manager.cc:359] Metadata directory not provided
I20260812 02:41:47.833238 582 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:41:47.833580 582 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:41:47.836752 582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786502156971023-7698-0/minicluster-data/master-0/data/instance:
uuid: "567ef627b5e74deab29b68a33b78adae"
format_stamp: "Formatted at 2026-08-12 02:41:47 on dist-test-slave-mn4r"
server_key: "80eecc24d266bf29e817d1bd2f0382f5"
server_key_iv: "2972b708ed20051471726b0089948cd3"
server_key_version: "encryptionkey@0"
I20260812 02:41:47.837567 582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786502156971023-7698-0/minicluster-data/master-0/wal/instance:
uuid: "567ef627b5e74deab29b68a33b78adae"
format_stamp: "Formatted at 2026-08-12 02:41:47 on dist-test-slave-mn4r"
server_key: "80eecc24d266bf29e817d1bd2f0382f5"
server_key_iv: "2972b708ed20051471726b0089948cd3"
server_key_version: "encryptionkey@0"
I20260812 02:41:47.843046 582 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260812 02:41:47.846565 598 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:41:47.847848 582 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260812 02:41:47.848047 582 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786502156971023-7698-0/minicluster-data/master-0/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786502156971023-7698-0/minicluster-data/master-0/wal
uuid: "567ef627b5e74deab29b68a33b78adae"
format_stamp: "Formatted at 2026-08-12 02:41:47 on dist-test-slave-mn4r"
server_key: "80eecc24d266bf29e817d1bd2f0382f5"
server_key_iv: "2972b708ed20051471726b0089948cd3"
server_key_version: "encryptionkey@0"
I20260812 02:41:47.848253 582 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786502156971023-7698-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786502156971023-7698-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786502156971023-7698-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:41:47.863322 582 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:41:47.867368 582 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:41:47.867681 582 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:41:47.886868 582 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44095
I20260812 02:41:47.886905 663 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44095 every 8 connection(s)
I20260812 02:41:47.888306 582 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
I20260812 02:41:47.890316 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 582
I20260812 02:41:47.890513 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786502156971023-7698-0/minicluster-data/master-0/wal/instance
I20260812 02:41:47.890969 7698 external_mini_cluster.cc:1428] Setting key aac4e60ef84c9503c23dfb970529a8df
I20260812 02:41:47.892657 664 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 12 02:41:47 dist-test-slave-mn4r krb5kdc[569](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502507, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 02:41:47.907064 664 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 567ef627b5e74deab29b68a33b78adae: Bootstrap starting.
I20260812 02:41:47.910465 664 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 567ef627b5e74deab29b68a33b78adae: Neither blocks nor log segments found. Creating new log.
I20260812 02:41:47.911777 664 log.cc:824] T 00000000000000000000000000000000 P 567ef627b5e74deab29b68a33b78adae: Log is configured to *not* fsync() on all Append() calls
I20260812 02:41:47.915102 664 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 567ef627b5e74deab29b68a33b78adae: No bootstrap required, opened a new log
I20260812 02:41:47.915593 667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:47.892886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.160:39035 (local address 127.0.0.1:44095)
0812 02:41:47.893707 (+ 821us) server_negotiation.cc:206] Beginning negotiation
0812 02:41:47.893723 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 02:41:47.893774 (+ 51us) server_negotiation.cc:407] Connection header received
0812 02:41:47.895968 (+ 2194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:41:47.895997 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:41:47.896533 (+ 536us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:41:47.897001 (+ 468us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:41:47.898367 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:47.900501 (+ 2134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:47.901774 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:47.902877 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:47.903098 (+ 221us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:47.907648 (+ 4550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:41:47.907656 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:41:47.907670 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:41:47.907710 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:41:47.910917 (+ 3207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:41:47.911964 (+ 1047us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:41:47.911979 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:41:47.911985 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:41:47.912169 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:41:47.912751 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:41:47.912759 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:41:47.912765 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:41:47.913385 (+ 620us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:41:47.913613 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0812 02:41:47.914232 (+ 619us) server_negotiation.cc:299] Negotiation successful
0812 02:41:47.914535 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":573,"thread_start_us":331,"threads_started":1}
I20260812 02:41:47.919456 7698 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 02:41:47.923177 664 raft_consensus.cc:348] T 00000000000000000000000000000000 P 567ef627b5e74deab29b68a33b78adae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "567ef627b5e74deab29b68a33b78adae" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44095 } }
I20260812 02:41:47.923573 664 raft_consensus.cc:374] T 00000000000000000000000000000000 P 567ef627b5e74deab29b68a33b78adae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:41:47.923688 664 raft_consensus.cc:729] T 00000000000000000000000000000000 P 567ef627b5e74deab29b68a33b78adae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 567ef627b5e74deab29b68a33b78adae, State: Initialized, Role: FOLLOWER
I20260812 02:41:47.924419 664 consensus_queue.cc:260] T 00000000000000000000000000000000 P 567ef627b5e74deab29b68a33b78adae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "567ef627b5e74deab29b68a33b78adae" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44095 } }
I20260812 02:41:47.924734 664 raft_consensus.cc:388] T 00000000000000000000000000000000 P 567ef627b5e74deab29b68a33b78adae [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 02:41:47.924908 664 raft_consensus.cc:482] T 00000000000000000000000000000000 P 567ef627b5e74deab29b68a33b78adae [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 02:41:47.925115 664 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 567ef627b5e74deab29b68a33b78adae [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:41:47.927423 664 raft_consensus.cc:504] T 00000000000000000000000000000000 P 567ef627b5e74deab29b68a33b78adae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "567ef627b5e74deab29b68a33b78adae" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44095 } }
I20260812 02:41:47.927928 664 leader_election.cc:302] T 00000000000000000000000000000000 P 567ef627b5e74deab29b68a33b78adae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 567ef627b5e74deab29b68a33b78adae; no voters:
I20260812 02:41:47.928644 664 leader_election.cc:288] T 00000000000000000000000000000000 P 567ef627b5e74deab29b68a33b78adae [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 02:41:47.928772 676 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 567ef627b5e74deab29b68a33b78adae [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:41:47.931375 664 sys_catalog.cc:564] T 00000000000000000000000000000000 P 567ef627b5e74deab29b68a33b78adae [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:41:47.935891 676 raft_consensus.cc:686] T 00000000000000000000000000000000 P 567ef627b5e74deab29b68a33b78adae [term 1 LEADER]: Becoming Leader. State: Replica: 567ef627b5e74deab29b68a33b78adae, State: Running, Role: LEADER
I20260812 02:41:47.936627 676 consensus_queue.cc:237] T 00000000000000000000000000000000 P 567ef627b5e74deab29b68a33b78adae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "567ef627b5e74deab29b68a33b78adae" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44095 } }
I20260812 02:41:47.938831 667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:47.923904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.160:53613 (local address 127.0.0.1:44095)
0812 02:41:47.924124 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0812 02:41:47.924134 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 02:41:47.924622 (+ 488us) server_negotiation.cc:407] Connection header received
0812 02:41:47.924830 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:41:47.924834 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:41:47.924935 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:41:47.925038 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:41:47.926562 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:47.927331 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:47.928654 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:47.929523 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:47.929695 (+ 172us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:47.932246 (+ 2551us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:41:47.932254 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:41:47.932260 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:41:47.932298 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:41:47.935716 (+ 3418us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:41:47.936562 (+ 846us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:41:47.936567 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:41:47.936571 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:41:47.936788 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:41:47.937269 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:41:47.937274 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:41:47.937278 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:41:47.937584 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:41:47.938033 (+ 449us) server_negotiation.cc:1035] Waiting for connection context
0812 02:41:47.938277 (+ 244us) server_negotiation.cc:299] Negotiation successful
0812 02:41:47.938463 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 02:41:47.947860 677 sys_catalog.cc:455] T 00000000000000000000000000000000 P 567ef627b5e74deab29b68a33b78adae [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "567ef627b5e74deab29b68a33b78adae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "567ef627b5e74deab29b68a33b78adae" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44095 } } }
I20260812 02:41:47.948403 677 sys_catalog.cc:458] T 00000000000000000000000000000000 P 567ef627b5e74deab29b68a33b78adae [sys.catalog]: This master's current role is: LEADER
I20260812 02:41:47.948021 678 sys_catalog.cc:455] T 00000000000000000000000000000000 P 567ef627b5e74deab29b68a33b78adae [sys.catalog]: SysCatalogTable state changed. Reason: New leader 567ef627b5e74deab29b68a33b78adae. Latest consensus state: current_term: 1 leader_uuid: "567ef627b5e74deab29b68a33b78adae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "567ef627b5e74deab29b68a33b78adae" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44095 } } }
I20260812 02:41:47.948945 678 sys_catalog.cc:458] T 00000000000000000000000000000000 P 567ef627b5e74deab29b68a33b78adae [sys.catalog]: This master's current role is: LEADER
I20260812 02:41:47.950351 693 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:41:47.955063 693 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 02:41:47.962829 693 catalog_manager.cc:1347] Generated new cluster ID: 4021c72b9c4e4dedb45a275d51628c12
I20260812 02:41:47.962945 693 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 02:41:47.975740 693 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 02:41:47.977593 693 catalog_manager.cc:1504] Loading token signing keys...
I20260812 02:41:47.987521 693 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 567ef627b5e74deab29b68a33b78adae: Generated new TSK 0
I20260812 02:41:47.988620 693 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 02:41:47.995471 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 582
I20260812 02:41:47.996902 656 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:41:48.088006 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 582
W20260812 02:41:48.112576 567 connection.cc:535] client connection to 127.0.0.1:44095 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44095): Connection reset by peer (error 104)
2026-08-12T02:41:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (2246 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786502156971023-7698-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 02:41:48 dist-test-slave-mn4r krb5kdc[702](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 02:41:48 dist-test-slave-mn4r krb5kdc[702](info): set up 2 sockets
Aug 12 02:41:48 dist-test-slave-mn4r krb5kdc[702](info): commencing operation
krb5kdc: starting...
W20260812 02:41:49.410667 7698 mini_kdc.cc:121] Time spent starting KDC: real 1.279s user 0.003s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786502156971023-7698-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786502156971023-7698-0/krb5kdc/test-user.keytab.
Aug 12 02:41:49 dist-test-slave-mn4r krb5kdc[702](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502509, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T02:41:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T02:41:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786502156971023-7698-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786502156971023-7698-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 02:41:49.749790 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786502156971023-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786502156971023-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786502156971023-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43457
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786502156971023-7698-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43457
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:41:50.110648 717 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:41:50.110965 717 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:41:50.111157 717 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:41:50.118614 717 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:41:50.118721 717 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:41:50.118824 717 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:41:50.118938 717 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:41:50.119021 717 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:41:50.129135 717 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786502156971023-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786502156971023-7698-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43457
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786502156971023-7698-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43457
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786502156971023-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:41:50.130579 717 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:41:50.132011 717 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:41:50.144136 725 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:41:50.144178 728 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:41:50.144235 724 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:41:50.145073 717 server_base.cc:1034] running on GCE node
I20260812 02:41:50.145780 717 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:41:50.147166 717 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:41:50.148406 717 hybrid_clock.cc:648] HybridClock initialized: now 1786502510148356 us; error 58 us; skew 500 ppm
I20260812 02:41:50.148779 717 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:41:50 dist-test-slave-mn4r krb5kdc[702](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502510, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:41:50.152149 717 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 02:41:50.153599 717 webserver.cc:460] Webserver started at http://127.0.0.1:46471/ using document root <none> and password file <none>
I20260812 02:41:50.154371 717 fs_manager.cc:359] Metadata directory not provided
I20260812 02:41:50.154522 717 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:41:50.154839 717 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:41:50.157805 717 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786502156971023-7698-0/minicluster-data/master-0/data/instance:
uuid: "a4d8beebf692485f8fc1ad719bc5c039"
format_stamp: "Formatted at 2026-08-12 02:41:50 on dist-test-slave-mn4r"
server_key: "7d01654a64fea5c48086d5b62b7880ad"
server_key_iv: "af5da426b04b4cb1ba4f9b8222b6aaed"
server_key_version: "encryptionkey@0"
I20260812 02:41:50.158952 717 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786502156971023-7698-0/minicluster-data/master-0/wal/instance:
uuid: "a4d8beebf692485f8fc1ad719bc5c039"
format_stamp: "Formatted at 2026-08-12 02:41:50 on dist-test-slave-mn4r"
server_key: "7d01654a64fea5c48086d5b62b7880ad"
server_key_iv: "af5da426b04b4cb1ba4f9b8222b6aaed"
server_key_version: "encryptionkey@0"
I20260812 02:41:50.164408 717 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260812 02:41:50.167932 735 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:41:50.169268 717 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 02:41:50.169466 717 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786502156971023-7698-0/minicluster-data/master-0/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786502156971023-7698-0/minicluster-data/master-0/wal
uuid: "a4d8beebf692485f8fc1ad719bc5c039"
format_stamp: "Formatted at 2026-08-12 02:41:50 on dist-test-slave-mn4r"
server_key: "7d01654a64fea5c48086d5b62b7880ad"
server_key_iv: "af5da426b04b4cb1ba4f9b8222b6aaed"
server_key_version: "encryptionkey@0"
I20260812 02:41:50.169670 717 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786502156971023-7698-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786502156971023-7698-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786502156971023-7698-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:41:50.190210 717 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:41:50.195192 717 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:41:50.195531 717 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:41:50.215819 717 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43457
I20260812 02:41:50.215852 794 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43457 every 8 connection(s)
I20260812 02:41:50.217332 717 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
I20260812 02:41:50.221155 796 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:41:50.221851 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 717
I20260812 02:41:50.222069 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786502156971023-7698-0/minicluster-data/master-0/wal/instance
I20260812 02:41:50.222774 7698 external_mini_cluster.cc:1428] Setting key 572b4f604ed48feeaaacff9c0152aa87
I20260812 02:41:50.235623 796 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4d8beebf692485f8fc1ad719bc5c039: Bootstrap starting.
I20260812 02:41:50.237473 800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:50.224832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.160:43263 (local address 127.0.0.1:43457)
0812 02:41:50.225711 (+ 879us) server_negotiation.cc:206] Beginning negotiation
0812 02:41:50.225723 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 02:41:50.225764 (+ 41us) server_negotiation.cc:407] Connection header received
0812 02:41:50.227930 (+ 2166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:41:50.227955 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:41:50.228433 (+ 478us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:41:50.228772 (+ 339us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:41:50.229401 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:50.230839 (+ 1438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:50.232810 (+ 1971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:50.233553 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:50.233698 (+ 145us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:50.235073 (+ 1375us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:41:50.235082 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:41:50.235095 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:41:50.235636 (+ 541us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:41:50.235745 (+ 109us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:41:50.235788 (+ 43us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:41:50.235923 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0812 02:41:50.236256 (+ 333us) server_negotiation.cc:299] Negotiation successful
0812 02:41:50.236453 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":593,"thread_start_us":397,"threads_started":1}
I20260812 02:41:50.239149 796 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a4d8beebf692485f8fc1ad719bc5c039: Neither blocks nor log segments found. Creating new log.
I20260812 02:41:50.240520 796 log.cc:824] T 00000000000000000000000000000000 P a4d8beebf692485f8fc1ad719bc5c039: Log is configured to *not* fsync() on all Append() calls
I20260812 02:41:50.241818 7698 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 02:41:50.243480 796 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4d8beebf692485f8fc1ad719bc5c039: No bootstrap required, opened a new log
I20260812 02:41:50.252558 796 raft_consensus.cc:348] T 00000000000000000000000000000000 P a4d8beebf692485f8fc1ad719bc5c039 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4d8beebf692485f8fc1ad719bc5c039" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43457 } }
I20260812 02:41:50.253145 796 raft_consensus.cc:374] T 00000000000000000000000000000000 P a4d8beebf692485f8fc1ad719bc5c039 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:41:50.253286 796 raft_consensus.cc:729] T 00000000000000000000000000000000 P a4d8beebf692485f8fc1ad719bc5c039 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4d8beebf692485f8fc1ad719bc5c039, State: Initialized, Role: FOLLOWER
I20260812 02:41:50.253657 800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:50.246421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.160:59157 (local address 127.0.0.1:43457)
0812 02:41:50.246613 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0812 02:41:50.246620 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:41:50.247088 (+ 468us) server_negotiation.cc:407] Connection header received
0812 02:41:50.247230 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:41:50.247235 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:41:50.247310 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:41:50.247421 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:41:50.247912 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:50.249166 (+ 1254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:50.250599 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:50.251324 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:50.251455 (+ 131us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:50.252742 (+ 1287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:41:50.252748 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:41:50.252752 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:41:50.252773 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:41:50.252838 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:41:50.252861 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:41:50.252992 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0812 02:41:50.253258 (+ 266us) server_negotiation.cc:299] Negotiation successful
0812 02:41:50.253319 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260812 02:41:50.254067 796 consensus_queue.cc:260] T 00000000000000000000000000000000 P a4d8beebf692485f8fc1ad719bc5c039 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4d8beebf692485f8fc1ad719bc5c039" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43457 } }
I20260812 02:41:50.254403 796 raft_consensus.cc:388] T 00000000000000000000000000000000 P a4d8beebf692485f8fc1ad719bc5c039 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 02:41:50.254580 796 raft_consensus.cc:482] T 00000000000000000000000000000000 P a4d8beebf692485f8fc1ad719bc5c039 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 02:41:50.254765 796 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a4d8beebf692485f8fc1ad719bc5c039 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:41:50.257210 796 raft_consensus.cc:504] T 00000000000000000000000000000000 P a4d8beebf692485f8fc1ad719bc5c039 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4d8beebf692485f8fc1ad719bc5c039" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43457 } }
I20260812 02:41:50.257774 796 leader_election.cc:302] T 00000000000000000000000000000000 P a4d8beebf692485f8fc1ad719bc5c039 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a4d8beebf692485f8fc1ad719bc5c039; no voters:
I20260812 02:41:50.258563 796 leader_election.cc:288] T 00000000000000000000000000000000 P a4d8beebf692485f8fc1ad719bc5c039 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 02:41:50.258677 809 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a4d8beebf692485f8fc1ad719bc5c039 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:41:50.259371 809 raft_consensus.cc:686] T 00000000000000000000000000000000 P a4d8beebf692485f8fc1ad719bc5c039 [term 1 LEADER]: Becoming Leader. State: Replica: a4d8beebf692485f8fc1ad719bc5c039, State: Running, Role: LEADER
I20260812 02:41:50.260025 809 consensus_queue.cc:237] T 00000000000000000000000000000000 P a4d8beebf692485f8fc1ad719bc5c039 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4d8beebf692485f8fc1ad719bc5c039" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43457 } }
I20260812 02:41:50.260964 796 sys_catalog.cc:564] T 00000000000000000000000000000000 P a4d8beebf692485f8fc1ad719bc5c039 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:41:50.263618 813 sys_catalog.cc:455] T 00000000000000000000000000000000 P a4d8beebf692485f8fc1ad719bc5c039 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a4d8beebf692485f8fc1ad719bc5c039. Latest consensus state: current_term: 1 leader_uuid: "a4d8beebf692485f8fc1ad719bc5c039" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4d8beebf692485f8fc1ad719bc5c039" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43457 } } }
I20260812 02:41:50.263705 810 sys_catalog.cc:455] T 00000000000000000000000000000000 P a4d8beebf692485f8fc1ad719bc5c039 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a4d8beebf692485f8fc1ad719bc5c039" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4d8beebf692485f8fc1ad719bc5c039" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43457 } } }
I20260812 02:41:50.264087 813 sys_catalog.cc:458] T 00000000000000000000000000000000 P a4d8beebf692485f8fc1ad719bc5c039 [sys.catalog]: This master's current role is: LEADER
I20260812 02:41:50.264135 810 sys_catalog.cc:458] T 00000000000000000000000000000000 P a4d8beebf692485f8fc1ad719bc5c039 [sys.catalog]: This master's current role is: LEADER
I20260812 02:41:50.265584 819 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:41:50.272364 819 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 02:41:50.282013 819 catalog_manager.cc:1347] Generated new cluster ID: 047d0e08a57442c895c236418a99ea9a
I20260812 02:41:50.282187 819 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 02:41:50.310665 819 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 02:41:50.312589 819 catalog_manager.cc:1504] Loading token signing keys...
I20260812 02:41:50.323825 819 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a4d8beebf692485f8fc1ad719bc5c039: Generated new TSK 0
I20260812 02:41:50.324867 819 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 02:41:50.334622 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 717
W20260812 02:41:50.335749 790 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.160:43263
W20260812 02:41:50.336695 7698 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 717: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 02:41:50.336890 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 717
W20260812 02:41:50.361737 700 connection.cc:535] client connection to 127.0.0.1:43457 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43457): Connection reset by peer (error 104)
2026-08-12T02:41:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (2249 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786502156971023-7698-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 02:41:50 dist-test-slave-mn4r krb5kdc[836](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 02:41:50 dist-test-slave-mn4r krb5kdc[836](info): set up 2 sockets
Aug 12 02:41:50 dist-test-slave-mn4r krb5kdc[836](info): commencing operation
krb5kdc: starting...
W20260812 02:41:51.584570 7698 mini_kdc.cc:121] Time spent starting KDC: real 1.203s user 0.007s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786502156971023-7698-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786502156971023-7698-0/krb5kdc/test-user.keytab.
Aug 12 02:41:51 dist-test-slave-mn4r krb5kdc[836](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502511, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T02:41:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T02:41:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786502156971023-7698-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786502156971023-7698-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 02:41:51.913703 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786502156971023-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786502156971023-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786502156971023-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32897
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786502156971023-7698-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:32897
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:41:52.264230 850 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:41:52.264559 850 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:41:52.264744 850 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:41:52.272125 850 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:41:52.272239 850 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:41:52.272367 850 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:41:52.272477 850 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:41:52.272559 850 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:41:52.282614 850 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786502156971023-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786502156971023-7698-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:32897
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786502156971023-7698-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32897
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786502156971023-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:41:52.284006 850 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:41:52.285418 850 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:41:52.297750 857 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:41:52.297806 860 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:41:52.297837 856 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:41:52.298475 850 server_base.cc:1034] running on GCE node
I20260812 02:41:52.299238 850 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:41:52.300632 850 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:41:52.301844 850 hybrid_clock.cc:648] HybridClock initialized: now 1786502512301787 us; error 48 us; skew 500 ppm
I20260812 02:41:52.302285 850 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:41:52 dist-test-slave-mn4r krb5kdc[836](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502512, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:41:52.305449 850 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 02:41:52.307029 850 webserver.cc:460] Webserver started at http://127.0.0.1:35605/ using document root <none> and password file <none>
I20260812 02:41:52.307772 850 fs_manager.cc:359] Metadata directory not provided
I20260812 02:41:52.307912 850 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:41:52.308246 850 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:41:52.311408 850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786502156971023-7698-0/minicluster-data/master-0/data/instance:
uuid: "d59a1bd952c744bb9301edb4c1477465"
format_stamp: "Formatted at 2026-08-12 02:41:52 on dist-test-slave-mn4r"
server_key: "c0c1b48baf218ebbb2e5911e882e5893"
server_key_iv: "3ac1bb755b7fe9189f2e76166602786e"
server_key_version: "encryptionkey@0"
I20260812 02:41:52.312189 850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786502156971023-7698-0/minicluster-data/master-0/wal/instance:
uuid: "d59a1bd952c744bb9301edb4c1477465"
format_stamp: "Formatted at 2026-08-12 02:41:52 on dist-test-slave-mn4r"
server_key: "c0c1b48baf218ebbb2e5911e882e5893"
server_key_iv: "3ac1bb755b7fe9189f2e76166602786e"
server_key_version: "encryptionkey@0"
I20260812 02:41:52.317490 850 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260812 02:41:52.320966 869 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:41:52.322362 850 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 02:41:52.322547 850 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786502156971023-7698-0/minicluster-data/master-0/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786502156971023-7698-0/minicluster-data/master-0/wal
uuid: "d59a1bd952c744bb9301edb4c1477465"
format_stamp: "Formatted at 2026-08-12 02:41:52 on dist-test-slave-mn4r"
server_key: "c0c1b48baf218ebbb2e5911e882e5893"
server_key_iv: "3ac1bb755b7fe9189f2e76166602786e"
server_key_version: "encryptionkey@0"
I20260812 02:41:52.322746 850 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786502156971023-7698-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786502156971023-7698-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786502156971023-7698-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:41:52.328655 850 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:41:52.332876 850 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:41:52.333196 850 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:41:52.352245 850 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:32897
I20260812 02:41:52.352347 930 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:32897 every 8 connection(s)
I20260812 02:41:52.353680 850 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
I20260812 02:41:52.356403 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 850
I20260812 02:41:52.356603 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786502156971023-7698-0/minicluster-data/master-0/wal/instance
I20260812 02:41:52.357092 7698 external_mini_cluster.cc:1428] Setting key eaeb9ea1850ba49198cfbb34a20472b9
I20260812 02:41:52.357694 931 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:41:52.366766 934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:52.359109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.160:33417 (local address 127.0.0.1:32897)
0812 02:41:52.359951 (+ 842us) server_negotiation.cc:206] Beginning negotiation
0812 02:41:52.359967 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 02:41:52.360018 (+ 51us) server_negotiation.cc:407] Connection header received
0812 02:41:52.362674 (+ 2656us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:41:52.362706 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:41:52.363181 (+ 475us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:41:52.363692 (+ 511us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:41:52.364733 (+ 1041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:41:52.364741 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:41:52.364758 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:41:52.364781 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:41:52.364852 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:41:52.364882 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:41:52.365119 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0812 02:41:52.365256 (+ 137us) server_negotiation.cc:299] Negotiation successful
0812 02:41:52.365473 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":583,"thread_start_us":350,"threads_started":1}
I20260812 02:41:52.370579 7698 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 02:41:52.372267 931 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d59a1bd952c744bb9301edb4c1477465: Bootstrap starting.
I20260812 02:41:52.375564 931 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d59a1bd952c744bb9301edb4c1477465: Neither blocks nor log segments found. Creating new log.
I20260812 02:41:52.376803 931 log.cc:824] T 00000000000000000000000000000000 P d59a1bd952c744bb9301edb4c1477465: Log is configured to *not* fsync() on all Append() calls
I20260812 02:41:52.377686 934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:52.375152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.160:45147 (local address 127.0.0.1:32897)
0812 02:41:52.375326 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0812 02:41:52.375332 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:41:52.375751 (+ 419us) server_negotiation.cc:407] Connection header received
0812 02:41:52.375913 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:41:52.375923 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:41:52.375996 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:41:52.376126 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:41:52.376995 (+ 869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:41:52.377001 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:41:52.377007 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:41:52.377029 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:41:52.377074 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:41:52.377089 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:41:52.377209 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0812 02:41:52.377338 (+ 129us) server_negotiation.cc:299] Negotiation successful
0812 02:41:52.377401 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260812 02:41:52.379813 931 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d59a1bd952c744bb9301edb4c1477465: No bootstrap required, opened a new log
I20260812 02:41:52.387001 931 raft_consensus.cc:348] T 00000000000000000000000000000000 P d59a1bd952c744bb9301edb4c1477465 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d59a1bd952c744bb9301edb4c1477465" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32897 } }
I20260812 02:41:52.387382 931 raft_consensus.cc:374] T 00000000000000000000000000000000 P d59a1bd952c744bb9301edb4c1477465 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:41:52.387493 931 raft_consensus.cc:729] T 00000000000000000000000000000000 P d59a1bd952c744bb9301edb4c1477465 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d59a1bd952c744bb9301edb4c1477465, State: Initialized, Role: FOLLOWER
I20260812 02:41:52.388044 931 consensus_queue.cc:260] T 00000000000000000000000000000000 P d59a1bd952c744bb9301edb4c1477465 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d59a1bd952c744bb9301edb4c1477465" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32897 } }
I20260812 02:41:52.388273 931 raft_consensus.cc:388] T 00000000000000000000000000000000 P d59a1bd952c744bb9301edb4c1477465 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 02:41:52.388423 931 raft_consensus.cc:482] T 00000000000000000000000000000000 P d59a1bd952c744bb9301edb4c1477465 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 02:41:52.388587 931 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d59a1bd952c744bb9301edb4c1477465 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:41:52.391235 931 raft_consensus.cc:504] T 00000000000000000000000000000000 P d59a1bd952c744bb9301edb4c1477465 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d59a1bd952c744bb9301edb4c1477465" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32897 } }
I20260812 02:41:52.391707 931 leader_election.cc:302] T 00000000000000000000000000000000 P d59a1bd952c744bb9301edb4c1477465 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d59a1bd952c744bb9301edb4c1477465; no voters:
I20260812 02:41:52.392287 931 leader_election.cc:288] T 00000000000000000000000000000000 P d59a1bd952c744bb9301edb4c1477465 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 02:41:52.392406 941 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d59a1bd952c744bb9301edb4c1477465 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:41:52.393106 941 raft_consensus.cc:686] T 00000000000000000000000000000000 P d59a1bd952c744bb9301edb4c1477465 [term 1 LEADER]: Becoming Leader. State: Replica: d59a1bd952c744bb9301edb4c1477465, State: Running, Role: LEADER
I20260812 02:41:52.393606 941 consensus_queue.cc:237] T 00000000000000000000000000000000 P d59a1bd952c744bb9301edb4c1477465 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d59a1bd952c744bb9301edb4c1477465" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32897 } }
I20260812 02:41:52.394419 931 sys_catalog.cc:564] T 00000000000000000000000000000000 P d59a1bd952c744bb9301edb4c1477465 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:41:52.396799 942 sys_catalog.cc:455] T 00000000000000000000000000000000 P d59a1bd952c744bb9301edb4c1477465 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d59a1bd952c744bb9301edb4c1477465" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d59a1bd952c744bb9301edb4c1477465" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32897 } } }
I20260812 02:41:52.397271 942 sys_catalog.cc:458] T 00000000000000000000000000000000 P d59a1bd952c744bb9301edb4c1477465 [sys.catalog]: This master's current role is: LEADER
I20260812 02:41:52.397833 943 sys_catalog.cc:455] T 00000000000000000000000000000000 P d59a1bd952c744bb9301edb4c1477465 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d59a1bd952c744bb9301edb4c1477465. Latest consensus state: current_term: 1 leader_uuid: "d59a1bd952c744bb9301edb4c1477465" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d59a1bd952c744bb9301edb4c1477465" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32897 } } }
I20260812 02:41:52.398211 943 sys_catalog.cc:458] T 00000000000000000000000000000000 P d59a1bd952c744bb9301edb4c1477465 [sys.catalog]: This master's current role is: LEADER
I20260812 02:41:52.403251 950 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:41:52.408330 950 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 02:41:52.416988 950 catalog_manager.cc:1347] Generated new cluster ID: 7ba10a125fbc43a3b8be9ab6e371e393
I20260812 02:41:52.417102 950 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 02:41:52.456048 950 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 02:41:52.456501 950 catalog_manager.cc:1504] Loading token signing keys...
I20260812 02:41:52.465345 950 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d59a1bd952c744bb9301edb4c1477465: Generated new TSK 0
I20260812 02:41:52.466246 950 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 02:41:52.475806 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 850
W20260812 02:41:52.477423 924 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.160:33417
W20260812 02:41:52.478185 7698 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 850: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 02:41:52.478329 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 850
2026-08-12T02:41:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (2140 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786502156971023-7698-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 02:41:52 dist-test-slave-mn4r krb5kdc[965](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 02:41:52 dist-test-slave-mn4r krb5kdc[965](info): set up 2 sockets
Aug 12 02:41:52 dist-test-slave-mn4r krb5kdc[965](info): commencing operation
krb5kdc: starting...
W20260812 02:41:53.715620 7698 mini_kdc.cc:121] Time spent starting KDC: real 1.193s user 0.000s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786502156971023-7698-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786502156971023-7698-0/krb5kdc/test-user.keytab.
Aug 12 02:41:53 dist-test-slave-mn4r krb5kdc[965](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502513, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T02:41:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T02:41:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786502156971023-7698-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786502156971023-7698-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 02:41:54.064932 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786502156971023-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786502156971023-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786502156971023-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36413
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786502156971023-7698-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36413
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:41:54.416308 982 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:41:54.416668 982 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:41:54.416852 982 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:41:54.424391 982 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:41:54.424559 982 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:41:54.424672 982 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:41:54.424785 982 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:41:54.424875 982 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:41:54.435039 982 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786502156971023-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786502156971023-7698-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36413
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786502156971023-7698-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36413
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786502156971023-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:41:54.436525 982 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:41:54.437970 982 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:41:54.450635 991 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:41:54.450697 988 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:41:54.450903 989 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:41:54.451275 982 server_base.cc:1034] running on GCE node
I20260812 02:41:54.452258 982 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:41:54.453715 982 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:41:54.454921 982 hybrid_clock.cc:648] HybridClock initialized: now 1786502514454886 us; error 46 us; skew 500 ppm
I20260812 02:41:54.455320 982 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:41:54 dist-test-slave-mn4r krb5kdc[965](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502514, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:41:54.462858 982 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 02:41:54.464501 982 webserver.cc:460] Webserver started at http://127.0.0.1:35473/ using document root <none> and password file <none>
I20260812 02:41:54.465245 982 fs_manager.cc:359] Metadata directory not provided
I20260812 02:41:54.465410 982 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:41:54.465834 982 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:41:54.469084 982 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786502156971023-7698-0/minicluster-data/master-0/data/instance:
uuid: "b5d4acc79eb84408845ee0869822cde9"
format_stamp: "Formatted at 2026-08-12 02:41:54 on dist-test-slave-mn4r"
server_key: "7acd4270f23c50a77fadd43ded012301"
server_key_iv: "febb6f485660568d6161a37b2b279c2b"
server_key_version: "encryptionkey@0"
I20260812 02:41:54.469885 982 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786502156971023-7698-0/minicluster-data/master-0/wal/instance:
uuid: "b5d4acc79eb84408845ee0869822cde9"
format_stamp: "Formatted at 2026-08-12 02:41:54 on dist-test-slave-mn4r"
server_key: "7acd4270f23c50a77fadd43ded012301"
server_key_iv: "febb6f485660568d6161a37b2b279c2b"
server_key_version: "encryptionkey@0"
I20260812 02:41:54.475291 982 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260812 02:41:54.478904 999 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:41:54.480238 982 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 02:41:54.480420 982 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786502156971023-7698-0/minicluster-data/master-0/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786502156971023-7698-0/minicluster-data/master-0/wal
uuid: "b5d4acc79eb84408845ee0869822cde9"
format_stamp: "Formatted at 2026-08-12 02:41:54 on dist-test-slave-mn4r"
server_key: "7acd4270f23c50a77fadd43ded012301"
server_key_iv: "febb6f485660568d6161a37b2b279c2b"
server_key_version: "encryptionkey@0"
I20260812 02:41:54.480623 982 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786502156971023-7698-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786502156971023-7698-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786502156971023-7698-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:41:54.500511 982 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:41:54.505371 982 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:41:54.505698 982 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:41:54.525744 982 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36413
I20260812 02:41:54.525782 1058 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36413 every 8 connection(s)
I20260812 02:41:54.527246 982 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
I20260812 02:41:54.528591 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 982
I20260812 02:41:54.528782 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786502156971023-7698-0/minicluster-data/master-0/wal/instance
I20260812 02:41:54.529371 7698 external_mini_cluster.cc:1428] Setting key 50e7685ad8167a8d5587fe17c72b092b
I20260812 02:41:54.532114 1059 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 12 02:41:54 dist-test-slave-mn4r krb5kdc[965](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502513, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 02:41:54.545490 1059 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5d4acc79eb84408845ee0869822cde9: Bootstrap starting.
I20260812 02:41:54.548770 1059 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b5d4acc79eb84408845ee0869822cde9: Neither blocks nor log segments found. Creating new log.
I20260812 02:41:54.549841 1059 log.cc:824] T 00000000000000000000000000000000 P b5d4acc79eb84408845ee0869822cde9: Log is configured to *not* fsync() on all Append() calls
I20260812 02:41:54.553239 1059 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5d4acc79eb84408845ee0869822cde9: No bootstrap required, opened a new log
I20260812 02:41:54.553625 1064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:54.531341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.160:43475 (local address 127.0.0.1:36413)
0812 02:41:54.532183 (+ 842us) server_negotiation.cc:206] Beginning negotiation
0812 02:41:54.532203 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0812 02:41:54.532256 (+ 53us) server_negotiation.cc:407] Connection header received
0812 02:41:54.533914 (+ 1658us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:41:54.533936 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:41:54.534412 (+ 476us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:41:54.534748 (+ 336us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:41:54.536560 (+ 1812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:54.537899 (+ 1339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:54.539730 (+ 1831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:54.540392 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:54.540516 (+ 124us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:54.545609 (+ 5093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:41:54.545622 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:41:54.545643 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:41:54.545695 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:41:54.548871 (+ 3176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:41:54.549692 (+ 821us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:41:54.549699 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:41:54.549704 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:41:54.549868 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:41:54.550517 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:41:54.550525 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:41:54.550530 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:41:54.551348 (+ 818us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:41:54.551591 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0812 02:41:54.552162 (+ 571us) server_negotiation.cc:299] Negotiation successful
0812 02:41:54.552466 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":585,"thread_start_us":369,"threads_started":1}
I20260812 02:41:54.557670 7698 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 02:41:54.563220 1059 raft_consensus.cc:348] T 00000000000000000000000000000000 P b5d4acc79eb84408845ee0869822cde9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5d4acc79eb84408845ee0869822cde9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36413 } }
I20260812 02:41:54.563653 1059 raft_consensus.cc:374] T 00000000000000000000000000000000 P b5d4acc79eb84408845ee0869822cde9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:41:54.563773 1059 raft_consensus.cc:729] T 00000000000000000000000000000000 P b5d4acc79eb84408845ee0869822cde9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5d4acc79eb84408845ee0869822cde9, State: Initialized, Role: FOLLOWER
I20260812 02:41:54.564385 1059 consensus_queue.cc:260] T 00000000000000000000000000000000 P b5d4acc79eb84408845ee0869822cde9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5d4acc79eb84408845ee0869822cde9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36413 } }
I20260812 02:41:54.564623 1059 raft_consensus.cc:388] T 00000000000000000000000000000000 P b5d4acc79eb84408845ee0869822cde9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 02:41:54.564776 1059 raft_consensus.cc:482] T 00000000000000000000000000000000 P b5d4acc79eb84408845ee0869822cde9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 02:41:54.564955 1059 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b5d4acc79eb84408845ee0869822cde9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:41:54.567282 1059 raft_consensus.cc:504] T 00000000000000000000000000000000 P b5d4acc79eb84408845ee0869822cde9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5d4acc79eb84408845ee0869822cde9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36413 } }
I20260812 02:41:54.567777 1059 leader_election.cc:302] T 00000000000000000000000000000000 P b5d4acc79eb84408845ee0869822cde9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b5d4acc79eb84408845ee0869822cde9; no voters:
I20260812 02:41:54.568482 1059 leader_election.cc:288] T 00000000000000000000000000000000 P b5d4acc79eb84408845ee0869822cde9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 02:41:54.568611 1074 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b5d4acc79eb84408845ee0869822cde9 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:41:54.570340 1074 raft_consensus.cc:686] T 00000000000000000000000000000000 P b5d4acc79eb84408845ee0869822cde9 [term 1 LEADER]: Becoming Leader. State: Replica: b5d4acc79eb84408845ee0869822cde9, State: Running, Role: LEADER
I20260812 02:41:54.570938 1074 consensus_queue.cc:237] T 00000000000000000000000000000000 P b5d4acc79eb84408845ee0869822cde9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5d4acc79eb84408845ee0869822cde9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36413 } }
I20260812 02:41:54.571165 1059 sys_catalog.cc:564] T 00000000000000000000000000000000 P b5d4acc79eb84408845ee0869822cde9 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:41:54.574131 1076 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5d4acc79eb84408845ee0869822cde9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b5d4acc79eb84408845ee0869822cde9. Latest consensus state: current_term: 1 leader_uuid: "b5d4acc79eb84408845ee0869822cde9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5d4acc79eb84408845ee0869822cde9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36413 } } }
I20260812 02:41:54.574666 1076 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5d4acc79eb84408845ee0869822cde9 [sys.catalog]: This master's current role is: LEADER
I20260812 02:41:54.574247 1075 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5d4acc79eb84408845ee0869822cde9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b5d4acc79eb84408845ee0869822cde9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5d4acc79eb84408845ee0869822cde9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36413 } } }
I20260812 02:41:54.575239 1075 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5d4acc79eb84408845ee0869822cde9 [sys.catalog]: This master's current role is: LEADER
I20260812 02:41:54.581709 1064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:54.563112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.160:43847 (local address 127.0.0.1:36413)
0812 02:41:54.563316 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0812 02:41:54.563322 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:41:54.563827 (+ 505us) server_negotiation.cc:407] Connection header received
0812 02:41:54.564016 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:41:54.564026 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:41:54.564148 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:41:54.564264 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:41:54.565870 (+ 1606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:54.566735 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:54.568077 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:54.568630 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:54.568785 (+ 155us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:54.571966 (+ 3181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:41:54.571971 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:41:54.571974 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:41:54.572003 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:41:54.577337 (+ 5334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:41:54.578602 (+ 1265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:41:54.578610 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:41:54.578618 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:41:54.578830 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:41:54.579541 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:41:54.579550 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:41:54.579556 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:41:54.580073 (+ 517us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:41:54.580456 (+ 383us) server_negotiation.cc:1035] Waiting for connection context
0812 02:41:54.580914 (+ 458us) server_negotiation.cc:299] Negotiation successful
0812 02:41:54.581166 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260812 02:41:54.581943 1080 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:41:54.587570 1080 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 02:41:54.597138 1080 catalog_manager.cc:1347] Generated new cluster ID: 7a023d03f9204e7fb842a885d342d80c
I20260812 02:41:54.597267 1080 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 02:41:54.612147 1080 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 02:41:54.613422 1080 catalog_manager.cc:1504] Loading token signing keys...
I20260812 02:41:54.621147 1080 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b5d4acc79eb84408845ee0869822cde9: Generated new TSK 0
I20260812 02:41:54.622004 1080 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 02:41:54.630729 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 982
I20260812 02:41:54.631948 1053 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:41:54.725857 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 982
W20260812 02:41:54.750434 963 connection.cc:535] client connection to 127.0.0.1:36413 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36413): Connection reset by peer (error 104)
2026-08-12T02:41:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (2252 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786502156971023-7698-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 02:41:54 dist-test-slave-mn4r krb5kdc[1100](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 02:41:54 dist-test-slave-mn4r krb5kdc[1100](info): set up 2 sockets
Aug 12 02:41:54 dist-test-slave-mn4r krb5kdc[1100](info): commencing operation
krb5kdc: starting...
W20260812 02:41:55.985029 7698 mini_kdc.cc:121] Time spent starting KDC: real 1.210s user 0.004s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786502156971023-7698-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786502156971023-7698-0/krb5kdc/test-user.keytab.
Aug 12 02:41:56 dist-test-slave-mn4r krb5kdc[1100](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502516, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T02:41:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T02:41:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786502156971023-7698-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786502156971023-7698-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 02:41:56.322248 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786502156971023-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786502156971023-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786502156971023-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38789
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786502156971023-7698-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38789
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:41:56.702371 1116 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:41:56.702688 1116 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:41:56.702885 1116 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:41:56.710379 1116 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:41:56.710505 1116 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:41:56.710587 1116 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:41:56.710721 1116 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:41:56.710786 1116 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:41:56.720912 1116 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786502156971023-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786502156971023-7698-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38789
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786502156971023-7698-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38789
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786502156971023-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:41:56.722388 1116 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:41:56.723780 1116 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:41:56.736138 1122 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:41:56.736191 1123 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:41:56.736214 1126 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:41:56.737402 1116 server_base.cc:1034] running on GCE node
I20260812 02:41:56.738404 1116 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:41:56.740036 1116 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:41:56.741317 1116 hybrid_clock.cc:648] HybridClock initialized: now 1786502516741265 us; error 44 us; skew 500 ppm
I20260812 02:41:56.741734 1116 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:41:56 dist-test-slave-mn4r krb5kdc[1100](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502516, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:41:56.745478 1116 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 02:41:56.747414 1116 webserver.cc:460] Webserver started at http://127.0.0.1:36539/ using document root <none> and password file <none>
I20260812 02:41:56.748250 1116 fs_manager.cc:359] Metadata directory not provided
I20260812 02:41:56.748435 1116 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:41:56.748786 1116 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:41:56.752295 1116 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786502156971023-7698-0/minicluster-data/master-0/data/instance:
uuid: "88c2fd59830e44d89a3565e1863956cc"
format_stamp: "Formatted at 2026-08-12 02:41:56 on dist-test-slave-mn4r"
server_key: "424cc82156b0daf29d9f1b056fefa1a0"
server_key_iv: "3073e90610ac7010ed044fcf9d0fc619"
server_key_version: "encryptionkey@0"
I20260812 02:41:56.753157 1116 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786502156971023-7698-0/minicluster-data/master-0/wal/instance:
uuid: "88c2fd59830e44d89a3565e1863956cc"
format_stamp: "Formatted at 2026-08-12 02:41:56 on dist-test-slave-mn4r"
server_key: "424cc82156b0daf29d9f1b056fefa1a0"
server_key_iv: "3073e90610ac7010ed044fcf9d0fc619"
server_key_version: "encryptionkey@0"
I20260812 02:41:56.758894 1116 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260812 02:41:56.762809 1135 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:41:56.764267 1116 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 02:41:56.764475 1116 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786502156971023-7698-0/minicluster-data/master-0/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786502156971023-7698-0/minicluster-data/master-0/wal
uuid: "88c2fd59830e44d89a3565e1863956cc"
format_stamp: "Formatted at 2026-08-12 02:41:56 on dist-test-slave-mn4r"
server_key: "424cc82156b0daf29d9f1b056fefa1a0"
server_key_iv: "3073e90610ac7010ed044fcf9d0fc619"
server_key_version: "encryptionkey@0"
I20260812 02:41:56.764694 1116 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786502156971023-7698-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786502156971023-7698-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786502156971023-7698-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:41:56.780629 1116 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:41:56.785423 1116 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:41:56.785749 1116 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:41:56.806030 1116 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38789
I20260812 02:41:56.806243 1197 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38789 every 8 connection(s)
I20260812 02:41:56.807683 1116 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
I20260812 02:41:56.811827 1198 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:41:56.817500 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 1116
I20260812 02:41:56.817688 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786502156971023-7698-0/minicluster-data/master-0/wal/instance
I20260812 02:41:56.818238 7698 external_mini_cluster.cc:1428] Setting key 6866e20b7c9af0d8b7b5312f45c58b8a
I20260812 02:41:56.824421 1198 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88c2fd59830e44d89a3565e1863956cc: Bootstrap starting.
I20260812 02:41:56.828655 1198 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 88c2fd59830e44d89a3565e1863956cc: Neither blocks nor log segments found. Creating new log.
I20260812 02:41:56.829766 1198 log.cc:824] T 00000000000000000000000000000000 P 88c2fd59830e44d89a3565e1863956cc: Log is configured to *not* fsync() on all Append() calls
I20260812 02:41:56.833674 1198 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88c2fd59830e44d89a3565e1863956cc: No bootstrap required, opened a new log
I20260812 02:41:56.835620 1201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:56.820527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.160:41269 (local address 127.0.0.1:38789)
0812 02:41:56.821680 (+ 1153us) server_negotiation.cc:206] Beginning negotiation
0812 02:41:56.821697 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 02:41:56.822129 (+ 432us) server_negotiation.cc:407] Connection header received
0812 02:41:56.824654 (+ 2525us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:41:56.824682 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:41:56.825223 (+ 541us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:41:56.825682 (+ 459us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:41:56.826269 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:56.828331 (+ 2062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:56.829729 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:56.830954 (+ 1225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:56.831127 (+ 173us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:56.832480 (+ 1353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:41:56.832491 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:41:56.832511 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:41:56.833299 (+ 788us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:41:56.833401 (+ 102us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:41:56.833441 (+ 40us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:41:56.833621 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0812 02:41:56.833912 (+ 291us) server_negotiation.cc:299] Negotiation successful
0812 02:41:56.834144 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":798,"thread_start_us":492,"threads_started":1}
I20260812 02:41:56.839954 7698 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 02:41:56.843627 1198 raft_consensus.cc:348] T 00000000000000000000000000000000 P 88c2fd59830e44d89a3565e1863956cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88c2fd59830e44d89a3565e1863956cc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38789 } }
I20260812 02:41:56.844022 1198 raft_consensus.cc:374] T 00000000000000000000000000000000 P 88c2fd59830e44d89a3565e1863956cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:41:56.844162 1198 raft_consensus.cc:729] T 00000000000000000000000000000000 P 88c2fd59830e44d89a3565e1863956cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88c2fd59830e44d89a3565e1863956cc, State: Initialized, Role: FOLLOWER
I20260812 02:41:56.844964 1198 consensus_queue.cc:260] T 00000000000000000000000000000000 P 88c2fd59830e44d89a3565e1863956cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88c2fd59830e44d89a3565e1863956cc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38789 } }
I20260812 02:41:56.845291 1198 raft_consensus.cc:388] T 00000000000000000000000000000000 P 88c2fd59830e44d89a3565e1863956cc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 02:41:56.845477 1198 raft_consensus.cc:482] T 00000000000000000000000000000000 P 88c2fd59830e44d89a3565e1863956cc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 02:41:56.845718 1198 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 88c2fd59830e44d89a3565e1863956cc [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:41:56.849282 1198 raft_consensus.cc:504] T 00000000000000000000000000000000 P 88c2fd59830e44d89a3565e1863956cc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88c2fd59830e44d89a3565e1863956cc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38789 } }
I20260812 02:41:56.849967 1198 leader_election.cc:302] T 00000000000000000000000000000000 P 88c2fd59830e44d89a3565e1863956cc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 88c2fd59830e44d89a3565e1863956cc; no voters:
I20260812 02:41:56.850829 1198 leader_election.cc:288] T 00000000000000000000000000000000 P 88c2fd59830e44d89a3565e1863956cc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 02:41:56.850914 1208 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 88c2fd59830e44d89a3565e1863956cc [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:41:56.852687 1208 raft_consensus.cc:686] T 00000000000000000000000000000000 P 88c2fd59830e44d89a3565e1863956cc [term 1 LEADER]: Becoming Leader. State: Replica: 88c2fd59830e44d89a3565e1863956cc, State: Running, Role: LEADER
I20260812 02:41:56.852919 1201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:56.844700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.160:44795 (local address 127.0.0.1:38789)
0812 02:41:56.844937 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0812 02:41:56.844948 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 02:41:56.845375 (+ 427us) server_negotiation.cc:407] Connection header received
0812 02:41:56.845634 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:41:56.845639 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:41:56.845723 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:41:56.845805 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:41:56.846518 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:56.847334 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:56.849451 (+ 2117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:41:56.849982 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:41:56.850148 (+ 166us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:41:56.851733 (+ 1585us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:41:56.851741 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:41:56.851746 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:41:56.851768 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:41:56.851845 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:41:56.851873 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:41:56.851981 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0812 02:41:56.852361 (+ 380us) server_negotiation.cc:299] Negotiation successful
0812 02:41:56.852427 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 02:41:56.853617 1208 consensus_queue.cc:237] T 00000000000000000000000000000000 P 88c2fd59830e44d89a3565e1863956cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88c2fd59830e44d89a3565e1863956cc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38789 } }
I20260812 02:41:56.853794 1198 sys_catalog.cc:564] T 00000000000000000000000000000000 P 88c2fd59830e44d89a3565e1863956cc [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:41:56.860495 1210 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88c2fd59830e44d89a3565e1863956cc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 88c2fd59830e44d89a3565e1863956cc. Latest consensus state: current_term: 1 leader_uuid: "88c2fd59830e44d89a3565e1863956cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88c2fd59830e44d89a3565e1863956cc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38789 } } }
I20260812 02:41:56.860489 1209 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88c2fd59830e44d89a3565e1863956cc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "88c2fd59830e44d89a3565e1863956cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88c2fd59830e44d89a3565e1863956cc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38789 } } }
I20260812 02:41:56.860988 1210 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88c2fd59830e44d89a3565e1863956cc [sys.catalog]: This master's current role is: LEADER
I20260812 02:41:56.861038 1209 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88c2fd59830e44d89a3565e1863956cc [sys.catalog]: This master's current role is: LEADER
I20260812 02:41:56.862128 1221 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:41:56.868383 1221 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 02:41:56.878883 1221 catalog_manager.cc:1347] Generated new cluster ID: 90709e4ec065446c8056f843bbcf6caa
I20260812 02:41:56.879005 1221 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 02:41:56.894166 1221 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 02:41:56.895654 1221 catalog_manager.cc:1504] Loading token signing keys...
I20260812 02:41:56.911338 1221 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 88c2fd59830e44d89a3565e1863956cc: Generated new TSK 0
I20260812 02:41:56.912365 1221 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 02:41:56.918798 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 1116
W20260812 02:41:56.920105 1193 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.160:41269
W20260812 02:41:56.920945 7698 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 1116: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 02:41:56.921111 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 1116
W20260812 02:41:56.945990 1098 connection.cc:535] client connection to 127.0.0.1:38789 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38789): Connection reset by peer (error 104)
2026-08-12T02:41:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (2190 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786502156971023-7698-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 02:41:57 dist-test-slave-mn4r krb5kdc[1232](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 02:41:57 dist-test-slave-mn4r krb5kdc[1232](info): set up 2 sockets
Aug 12 02:41:57 dist-test-slave-mn4r krb5kdc[1232](info): commencing operation
krb5kdc: starting...
W20260812 02:41:58.157424 7698 mini_kdc.cc:121] Time spent starting KDC: real 1.192s user 0.001s sys 0.048s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786502156971023-7698-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786502156971023-7698-0/krb5kdc/test-user.keytab.
Aug 12 02:41:58 dist-test-slave-mn4r krb5kdc[1232](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502518, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T02:41:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T02:41:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786502156971023-7698-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786502156971023-7698-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 02:41:58.494870 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786502156971023-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786502156971023-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786502156971023-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37271
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786502156971023-7698-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42279
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37271
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:41:58.846968 1250 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:41:58.847272 1250 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:41:58.847465 1250 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:41:58.854724 1250 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:41:58.854830 1250 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:41:58.854946 1250 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:41:58.855051 1250 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:41:58.855134 1250 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:41:58.865164 1250 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42279
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786502156971023-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786502156971023-7698-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37271
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786502156971023-7698-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37271
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786502156971023-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:41:58.866657 1250 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:41:58.868070 1250 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:41:58.882678 1258 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:41:58.882735 1261 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:41:58.882761 1257 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:41:58.883646 1250 server_base.cc:1034] running on GCE node
I20260812 02:41:58.884503 1250 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:41:58.886197 1250 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:41:58.887473 1250 hybrid_clock.cc:648] HybridClock initialized: now 1786502518887429 us; error 56 us; skew 500 ppm
I20260812 02:41:58.887905 1250 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:41:58 dist-test-slave-mn4r krb5kdc[1232](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502518, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:41:58.891476 1250 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 02:41:58.893100 1250 webserver.cc:460] Webserver started at http://127.0.0.1:40167/ using document root <none> and password file <none>
I20260812 02:41:58.893867 1250 fs_manager.cc:359] Metadata directory not provided
I20260812 02:41:58.894044 1250 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:41:58.894457 1250 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:41:58.898006 1250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786502156971023-7698-0/minicluster-data/master-0/data/instance:
uuid: "f3036fe1ad9e4f9eb24942b4d8023e6a"
format_stamp: "Formatted at 2026-08-12 02:41:58 on dist-test-slave-mn4r"
server_key: "46cbf956fcfa6336d14688a554e96888"
server_key_iv: "3e59a76936a3123b9c7997c8c538e58d"
server_key_version: "encryptionkey@0"
I20260812 02:41:58.898983 1250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786502156971023-7698-0/minicluster-data/master-0/wal/instance:
uuid: "f3036fe1ad9e4f9eb24942b4d8023e6a"
format_stamp: "Formatted at 2026-08-12 02:41:58 on dist-test-slave-mn4r"
server_key: "46cbf956fcfa6336d14688a554e96888"
server_key_iv: "3e59a76936a3123b9c7997c8c538e58d"
server_key_version: "encryptionkey@0"
I20260812 02:41:58.905764 1250 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260812 02:41:58.913045 1270 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:41:58.915011 1250 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.005s sys 0.001s
I20260812 02:41:58.915231 1250 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786502156971023-7698-0/minicluster-data/master-0/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786502156971023-7698-0/minicluster-data/master-0/wal
uuid: "f3036fe1ad9e4f9eb24942b4d8023e6a"
format_stamp: "Formatted at 2026-08-12 02:41:58 on dist-test-slave-mn4r"
server_key: "46cbf956fcfa6336d14688a554e96888"
server_key_iv: "3e59a76936a3123b9c7997c8c538e58d"
server_key_version: "encryptionkey@0"
I20260812 02:41:58.915454 1250 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786502156971023-7698-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786502156971023-7698-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786502156971023-7698-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:41:58.921736 1250 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:41:58.926017 1250 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:41:58.926370 1250 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:41:58.945365 1250 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37271
I20260812 02:41:58.945451 1330 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37271 every 8 connection(s)
I20260812 02:41:58.946988 1250 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
I20260812 02:41:58.947871 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 1250
I20260812 02:41:58.948078 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786502156971023-7698-0/minicluster-data/master-0/wal/instance
I20260812 02:41:58.948546 7698 external_mini_cluster.cc:1428] Setting key 6ce1d37cd6d0491cfb6ca28f7ec342a2
I20260812 02:41:58.951610 1331 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:41:58.957026 1334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:58.950558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.160:42405 (local address 127.0.0.1:37271)
0812 02:41:58.951360 (+ 802us) server_negotiation.cc:206] Beginning negotiation
0812 02:41:58.951373 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 02:41:58.951416 (+ 43us) server_negotiation.cc:407] Connection header received
0812 02:41:58.953222 (+ 1806us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:41:58.953247 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:41:58.953598 (+ 351us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:41:58.954047 (+ 449us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:41:58.955241 (+ 1194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:41:58.955248 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:41:58.955262 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:41:58.955276 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:41:58.955361 (+ 85us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:41:58.955382 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:41:58.955644 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0812 02:41:58.955807 (+ 163us) server_negotiation.cc:299] Negotiation successful
0812 02:41:58.956007 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":570,"thread_start_us":341,"threads_started":1}
I20260812 02:41:58.961046 7698 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 02:41:58.963645 1331 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3036fe1ad9e4f9eb24942b4d8023e6a: Bootstrap starting.
I20260812 02:41:58.967229 1331 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f3036fe1ad9e4f9eb24942b4d8023e6a: Neither blocks nor log segments found. Creating new log.
I20260812 02:41:58.968451 1331 log.cc:824] T 00000000000000000000000000000000 P f3036fe1ad9e4f9eb24942b4d8023e6a: Log is configured to *not* fsync() on all Append() calls
I20260812 02:41:58.968793 1334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:41:58.965868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.160:34113 (local address 127.0.0.1:37271)
0812 02:41:58.966039 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0812 02:41:58.966044 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:41:58.966619 (+ 575us) server_negotiation.cc:407] Connection header received
0812 02:41:58.966832 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:41:58.966840 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:41:58.966900 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:41:58.967015 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:41:58.968072 (+ 1057us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:41:58.968078 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:41:58.968083 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:41:58.968109 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:41:58.968152 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:41:58.968170 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:41:58.968310 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0812 02:41:58.968413 (+ 103us) server_negotiation.cc:299] Negotiation successful
0812 02:41:58.968486 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 02:41:58.971870 1331 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3036fe1ad9e4f9eb24942b4d8023e6a: No bootstrap required, opened a new log
I20260812 02:41:58.979485 1331 raft_consensus.cc:348] T 00000000000000000000000000000000 P f3036fe1ad9e4f9eb24942b4d8023e6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3036fe1ad9e4f9eb24942b4d8023e6a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37271 } }
I20260812 02:41:58.979948 1331 raft_consensus.cc:374] T 00000000000000000000000000000000 P f3036fe1ad9e4f9eb24942b4d8023e6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:41:58.980031 1331 raft_consensus.cc:729] T 00000000000000000000000000000000 P f3036fe1ad9e4f9eb24942b4d8023e6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3036fe1ad9e4f9eb24942b4d8023e6a, State: Initialized, Role: FOLLOWER
I20260812 02:41:58.980614 1331 consensus_queue.cc:260] T 00000000000000000000000000000000 P f3036fe1ad9e4f9eb24942b4d8023e6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3036fe1ad9e4f9eb24942b4d8023e6a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37271 } }
I20260812 02:41:58.980818 1331 raft_consensus.cc:388] T 00000000000000000000000000000000 P f3036fe1ad9e4f9eb24942b4d8023e6a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 02:41:58.980919 1331 raft_consensus.cc:482] T 00000000000000000000000000000000 P f3036fe1ad9e4f9eb24942b4d8023e6a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 02:41:58.981045 1331 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f3036fe1ad9e4f9eb24942b4d8023e6a [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:41:58.983289 1331 raft_consensus.cc:504] T 00000000000000000000000000000000 P f3036fe1ad9e4f9eb24942b4d8023e6a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3036fe1ad9e4f9eb24942b4d8023e6a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37271 } }
I20260812 02:41:58.983749 1331 leader_election.cc:302] T 00000000000000000000000000000000 P f3036fe1ad9e4f9eb24942b4d8023e6a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f3036fe1ad9e4f9eb24942b4d8023e6a; no voters:
I20260812 02:41:58.984334 1331 leader_election.cc:288] T 00000000000000000000000000000000 P f3036fe1ad9e4f9eb24942b4d8023e6a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 02:41:58.984539 1341 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f3036fe1ad9e4f9eb24942b4d8023e6a [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:41:58.985330 1341 raft_consensus.cc:686] T 00000000000000000000000000000000 P f3036fe1ad9e4f9eb24942b4d8023e6a [term 1 LEADER]: Becoming Leader. State: Replica: f3036fe1ad9e4f9eb24942b4d8023e6a, State: Running, Role: LEADER
I20260812 02:41:58.986006 1341 consensus_queue.cc:237] T 00000000000000000000000000000000 P f3036fe1ad9e4f9eb24942b4d8023e6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3036fe1ad9e4f9eb24942b4d8023e6a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37271 } }
I20260812 02:41:58.986646 1331 sys_catalog.cc:564] T 00000000000000000000000000000000 P f3036fe1ad9e4f9eb24942b4d8023e6a [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:41:58.989490 1342 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3036fe1ad9e4f9eb24942b4d8023e6a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f3036fe1ad9e4f9eb24942b4d8023e6a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3036fe1ad9e4f9eb24942b4d8023e6a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37271 } } }
I20260812 02:41:58.989697 1343 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3036fe1ad9e4f9eb24942b4d8023e6a [sys.catalog]: SysCatalogTable state changed. Reason: New leader f3036fe1ad9e4f9eb24942b4d8023e6a. Latest consensus state: current_term: 1 leader_uuid: "f3036fe1ad9e4f9eb24942b4d8023e6a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3036fe1ad9e4f9eb24942b4d8023e6a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37271 } } }
I20260812 02:41:58.990015 1342 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3036fe1ad9e4f9eb24942b4d8023e6a [sys.catalog]: This master's current role is: LEADER
I20260812 02:41:58.990041 1343 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3036fe1ad9e4f9eb24942b4d8023e6a [sys.catalog]: This master's current role is: LEADER
I20260812 02:41:58.995179 1351 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:41:59.001344 1351 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 02:41:59.010757 1351 catalog_manager.cc:1347] Generated new cluster ID: 15fdefabe74e4243b62b846f3efa2792
I20260812 02:41:59.010903 1351 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 02:41:59.065958 1351 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 02:41:59.066473 1351 catalog_manager.cc:1504] Loading token signing keys...
I20260812 02:41:59.072265 1351 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f3036fe1ad9e4f9eb24942b4d8023e6a: Generated new TSK 0
I20260812 02:41:59.073055 1351 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 02:41:59.079797 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 1250
W20260812 02:41:59.081223 1325 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.160:42405
W20260812 02:41:59.082159 7698 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 1250: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 02:41:59.082336 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 1250
2026-08-12T02:41:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (2162 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786502156971023-7698-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 02:41:59 dist-test-slave-mn4r krb5kdc[1369](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 02:41:59 dist-test-slave-mn4r krb5kdc[1369](info): set up 2 sockets
Aug 12 02:41:59 dist-test-slave-mn4r krb5kdc[1369](info): commencing operation
krb5kdc: starting...
W20260812 02:42:00.326303 7698 mini_kdc.cc:121] Time spent starting KDC: real 1.198s user 0.000s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786502156971023-7698-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786502156971023-7698-0/krb5kdc/test-user.keytab.
Aug 12 02:42:00 dist-test-slave-mn4r krb5kdc[1369](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502520, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T02:42:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T02:42:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.132.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
I20260812 02:42:00.669862 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786502156971023-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786502156971023-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786502156971023-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:36769
--webserver_interface=127.7.132.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:46087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.132.190:36769
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:42:01.017975 1385 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:42:01.018292 1385 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:42:01.018474 1385 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:42:01.025844 1385 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:42:01.025964 1385 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:42:01.026103 1385 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:42:01.026218 1385 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:42:01.026297 1385 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:42:01.036428 1385 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:46087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786502156971023-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786502156971023-7698-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.132.190:36769
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:36769
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.132.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786502156971023-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:42:01.037828 1385 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:42:01.039278 1385 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:42:01.051294 1396 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:42:01.051573 1394 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:42:01.051578 1393 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:42:01.052363 1385 server_base.cc:1034] running on GCE node
I20260812 02:42:01.053223 1385 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:42:01.054699 1385 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:42:01.055905 1385 hybrid_clock.cc:648] HybridClock initialized: now 1786502521055827 us; error 65 us; skew 500 ppm
I20260812 02:42:01.056317 1385 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:42:01 dist-test-slave-mn4r krb5kdc[1369](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502521, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:42:01.059695 1385 init.cc:377] Logged in from keytab as kudu/127.7.132.190@KRBTEST.COM (short username kudu)
I20260812 02:42:01.061190 1385 webserver.cc:460] Webserver started at http://127.7.132.190:45513/ using document root <none> and password file <none>
I20260812 02:42:01.061950 1385 fs_manager.cc:359] Metadata directory not provided
I20260812 02:42:01.062070 1385 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:42:01.062423 1385 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:42:01.065653 1385 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786502156971023-7698-0/minicluster-data/master-0/data/instance:
uuid: "fbe1df585b8041efa6b31a04e4040d6b"
format_stamp: "Formatted at 2026-08-12 02:42:01 on dist-test-slave-mn4r"
server_key: "0a7613990951948330fff5e607c5bf0b"
server_key_iv: "124eb6a6013ede5b498699010800d167"
server_key_version: "encryptionkey@0"
I20260812 02:42:01.066510 1385 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786502156971023-7698-0/minicluster-data/master-0/wal/instance:
uuid: "fbe1df585b8041efa6b31a04e4040d6b"
format_stamp: "Formatted at 2026-08-12 02:42:01 on dist-test-slave-mn4r"
server_key: "0a7613990951948330fff5e607c5bf0b"
server_key_iv: "124eb6a6013ede5b498699010800d167"
server_key_version: "encryptionkey@0"
I20260812 02:42:01.071798 1385 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.001s
I20260812 02:42:01.075598 1406 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:42:01.077057 1385 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 02:42:01.077275 1385 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786502156971023-7698-0/minicluster-data/master-0/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786502156971023-7698-0/minicluster-data/master-0/wal
uuid: "fbe1df585b8041efa6b31a04e4040d6b"
format_stamp: "Formatted at 2026-08-12 02:42:01 on dist-test-slave-mn4r"
server_key: "0a7613990951948330fff5e607c5bf0b"
server_key_iv: "124eb6a6013ede5b498699010800d167"
server_key_version: "encryptionkey@0"
I20260812 02:42:01.077486 1385 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786502156971023-7698-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786502156971023-7698-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786502156971023-7698-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:42:01.095899 1385 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:42:01.100528 1385 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:42:01.100860 1385 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:42:01.119755 1385 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.190:36769
I20260812 02:42:01.119781 1476 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.190:36769 every 8 connection(s)
I20260812 02:42:01.121263 1385 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
I20260812 02:42:01.124127 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 1385
I20260812 02:42:01.124330 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786502156971023-7698-0/minicluster-data/master-0/wal/instance
I20260812 02:42:01.124781 7698 external_mini_cluster.cc:1428] Setting key 205c39b3237bbea91ad5dfcc2def9521
I20260812 02:42:01.125476 1478 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:42:01.140259 1478 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fbe1df585b8041efa6b31a04e4040d6b: Bootstrap starting.
Aug 12 02:42:01 dist-test-slave-mn4r krb5kdc[1369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502520, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 02:42:01.144309 1478 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fbe1df585b8041efa6b31a04e4040d6b: Neither blocks nor log segments found. Creating new log.
I20260812 02:42:01.145735 1478 log.cc:824] T 00000000000000000000000000000000 P fbe1df585b8041efa6b31a04e4040d6b: Log is configured to *not* fsync() on all Append() calls
I20260812 02:42:01.149435 1478 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fbe1df585b8041efa6b31a04e4040d6b: No bootstrap required, opened a new log
I20260812 02:42:01.151222 1483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:01.126797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32851 (local address 127.7.132.190:36769)
0812 02:42:01.127642 (+ 845us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:01.127658 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:01.127709 (+ 51us) server_negotiation.cc:407] Connection header received
0812 02:42:01.130412 (+ 2703us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:01.130442 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:01.131012 (+ 570us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:01.131487 (+ 475us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:01.132909 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:01.134955 (+ 2046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:01.136357 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:01.137496 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:01.137707 (+ 211us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:01.142102 (+ 4395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:42:01.142116 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:42:01.142141 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:42:01.142208 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:42:01.146232 (+ 4024us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:01.147347 (+ 1115us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:01.147357 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:01.147367 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:01.147569 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:01.148019 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:01.148023 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:01.148026 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:01.148743 (+ 717us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:42:01.149084 (+ 341us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:01.149477 (+ 393us) server_negotiation.cc:299] Negotiation successful
0812 02:42:01.149815 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":571,"thread_start_us":332,"threads_started":1}
I20260812 02:42:01.155579 7698 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 02:42:01.160215 1478 raft_consensus.cc:348] T 00000000000000000000000000000000 P fbe1df585b8041efa6b31a04e4040d6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbe1df585b8041efa6b31a04e4040d6b" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 36769 } }
I20260812 02:42:01.160693 1478 raft_consensus.cc:374] T 00000000000000000000000000000000 P fbe1df585b8041efa6b31a04e4040d6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:42:01.160806 1478 raft_consensus.cc:729] T 00000000000000000000000000000000 P fbe1df585b8041efa6b31a04e4040d6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbe1df585b8041efa6b31a04e4040d6b, State: Initialized, Role: FOLLOWER
I20260812 02:42:01.161546 1478 consensus_queue.cc:260] T 00000000000000000000000000000000 P fbe1df585b8041efa6b31a04e4040d6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbe1df585b8041efa6b31a04e4040d6b" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 36769 } }
I20260812 02:42:01.161880 1478 raft_consensus.cc:388] T 00000000000000000000000000000000 P fbe1df585b8041efa6b31a04e4040d6b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 02:42:01.162110 1478 raft_consensus.cc:482] T 00000000000000000000000000000000 P fbe1df585b8041efa6b31a04e4040d6b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 02:42:01.162320 1478 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fbe1df585b8041efa6b31a04e4040d6b [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:42:01.165416 1478 raft_consensus.cc:504] T 00000000000000000000000000000000 P fbe1df585b8041efa6b31a04e4040d6b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbe1df585b8041efa6b31a04e4040d6b" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 36769 } }
I20260812 02:42:01.166045 1478 leader_election.cc:302] T 00000000000000000000000000000000 P fbe1df585b8041efa6b31a04e4040d6b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fbe1df585b8041efa6b31a04e4040d6b; no voters:
I20260812 02:42:01.166786 1478 leader_election.cc:288] T 00000000000000000000000000000000 P fbe1df585b8041efa6b31a04e4040d6b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 02:42:01.166957 1493 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fbe1df585b8041efa6b31a04e4040d6b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:42:01.169214 1493 raft_consensus.cc:686] T 00000000000000000000000000000000 P fbe1df585b8041efa6b31a04e4040d6b [term 1 LEADER]: Becoming Leader. State: Replica: fbe1df585b8041efa6b31a04e4040d6b, State: Running, Role: LEADER
I20260812 02:42:01.169368 1478 sys_catalog.cc:564] T 00000000000000000000000000000000 P fbe1df585b8041efa6b31a04e4040d6b [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:42:01.169811 1493 consensus_queue.cc:237] T 00000000000000000000000000000000 P fbe1df585b8041efa6b31a04e4040d6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbe1df585b8041efa6b31a04e4040d6b" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 36769 } }
I20260812 02:42:01.173309 1496 sys_catalog.cc:455] T 00000000000000000000000000000000 P fbe1df585b8041efa6b31a04e4040d6b [sys.catalog]: SysCatalogTable state changed. Reason: New leader fbe1df585b8041efa6b31a04e4040d6b. Latest consensus state: current_term: 1 leader_uuid: "fbe1df585b8041efa6b31a04e4040d6b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbe1df585b8041efa6b31a04e4040d6b" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 36769 } } }
I20260812 02:42:01.173739 1496 sys_catalog.cc:458] T 00000000000000000000000000000000 P fbe1df585b8041efa6b31a04e4040d6b [sys.catalog]: This master's current role is: LEADER
I20260812 02:42:01.174407 1495 sys_catalog.cc:455] T 00000000000000000000000000000000 P fbe1df585b8041efa6b31a04e4040d6b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fbe1df585b8041efa6b31a04e4040d6b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbe1df585b8041efa6b31a04e4040d6b" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 36769 } } }
I20260812 02:42:01.174731 1495 sys_catalog.cc:458] T 00000000000000000000000000000000 P fbe1df585b8041efa6b31a04e4040d6b [sys.catalog]: This master's current role is: LEADER
I20260812 02:42:01.176934 1502 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:42:01.182672 1483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:01.160942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44351 (local address 127.7.132.190:36769)
0812 02:42:01.161115 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:01.161121 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:01.161758 (+ 637us) server_negotiation.cc:407] Connection header received
0812 02:42:01.162012 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:01.162020 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:01.162133 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:01.162262 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:01.164218 (+ 1956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:01.165001 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:01.166879 (+ 1878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:01.167772 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:01.167952 (+ 180us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:01.171579 (+ 3627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:42:01.171589 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:42:01.171597 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:42:01.171657 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:42:01.176748 (+ 5091us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:01.178070 (+ 1322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:01.178103 (+ 33us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:01.178109 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:01.178306 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:01.179035 (+ 729us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:01.179042 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:01.179048 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:01.179638 (+ 590us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:42:01.180039 (+ 401us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:01.181809 (+ 1770us) server_negotiation.cc:299] Negotiation successful
0812 02:42:01.182155 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 02:42:01.183027 1502 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 02:42:01.192381 1502 catalog_manager.cc:1347] Generated new cluster ID: 4e4356560112453c8a82a463f5e008ab
I20260812 02:42:01.192515 1502 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 02:42:01.205852 1502 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 02:42:01.207736 1502 catalog_manager.cc:1504] Loading token signing keys...
I20260812 02:42:01.225714 1502 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fbe1df585b8041efa6b31a04e4040d6b: Generated new TSK 0
I20260812 02:42:01.226991 1502 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 02:42:01.244755 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 1385
I20260812 02:42:01.246724 1470 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:42:01.341090 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 1385
W20260812 02:42:01.366039 1366 connection.cc:535] client connection to 127.7.132.190:36769 recv error: Network error: failed to read from TLS socket (remote: 127.7.132.190:36769): Connection reset by peer (error 104)
2026-08-12T02:42:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (2257 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786502156971023-7698-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 02:42:01 dist-test-slave-mn4r krb5kdc[1521](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 02:42:01 dist-test-slave-mn4r krb5kdc[1521](info): set up 2 sockets
Aug 12 02:42:01 dist-test-slave-mn4r krb5kdc[1521](info): commencing operation
krb5kdc: starting...
W20260812 02:42:02.590404 7698 mini_kdc.cc:121] Time spent starting KDC: real 1.205s user 0.000s sys 0.048s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786502156971023-7698-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786502156971023-7698-0/krb5kdc/test-user.keytab.
Aug 12 02:42:02 dist-test-slave-mn4r krb5kdc[1521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502522, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T02:42:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T02:42:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.132.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
I20260812 02:42:02.930049 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786502156971023-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786502156971023-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786502156971023-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:40183
--webserver_interface=127.7.132.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:33099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.132.190:40183
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:42:03.284778 1536 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:42:03.285194 1536 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:42:03.285372 1536 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:42:03.292639 1536 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:42:03.292809 1536 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:42:03.292925 1536 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:42:03.293046 1536 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:42:03.293136 1536 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:42:03.303400 1536 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:33099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786502156971023-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786502156971023-7698-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.132.190:40183
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:40183
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.132.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786502156971023-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:42:03.304785 1536 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:42:03.306195 1536 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:42:03.318779 1544 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:42:03.318848 1547 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:42:03.318816 1545 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:42:03.320050 1536 server_base.cc:1034] running on GCE node
I20260812 02:42:03.320919 1536 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:42:03.322626 1536 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:42:03.323861 1536 hybrid_clock.cc:648] HybridClock initialized: now 1786502523323796 us; error 56 us; skew 500 ppm
I20260812 02:42:03.324266 1536 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:42:03 dist-test-slave-mn4r krb5kdc[1521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502523, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:42:03.327656 1536 init.cc:377] Logged in from keytab as kudu/127.7.132.190@KRBTEST.COM (short username kudu)
I20260812 02:42:03.329087 1536 webserver.cc:460] Webserver started at http://127.7.132.190:35751/ using document root <none> and password file <none>
I20260812 02:42:03.329741 1536 fs_manager.cc:359] Metadata directory not provided
I20260812 02:42:03.329841 1536 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:42:03.330188 1536 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:42:03.333218 1536 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786502156971023-7698-0/minicluster-data/master-0/data/instance:
uuid: "609672c317804723befcd0cf7c89114d"
format_stamp: "Formatted at 2026-08-12 02:42:03 on dist-test-slave-mn4r"
server_key: "f39612a32d7ced4ed2eb30fa1f05d24f"
server_key_iv: "a5126c385b5f54cb0c08467198c6ce04"
server_key_version: "encryptionkey@0"
I20260812 02:42:03.333945 1536 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786502156971023-7698-0/minicluster-data/master-0/wal/instance:
uuid: "609672c317804723befcd0cf7c89114d"
format_stamp: "Formatted at 2026-08-12 02:42:03 on dist-test-slave-mn4r"
server_key: "f39612a32d7ced4ed2eb30fa1f05d24f"
server_key_iv: "a5126c385b5f54cb0c08467198c6ce04"
server_key_version: "encryptionkey@0"
I20260812 02:42:03.339228 1536 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260812 02:42:03.342868 1556 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:42:03.344537 1536 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 02:42:03.344731 1536 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786502156971023-7698-0/minicluster-data/master-0/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786502156971023-7698-0/minicluster-data/master-0/wal
uuid: "609672c317804723befcd0cf7c89114d"
format_stamp: "Formatted at 2026-08-12 02:42:03 on dist-test-slave-mn4r"
server_key: "f39612a32d7ced4ed2eb30fa1f05d24f"
server_key_iv: "a5126c385b5f54cb0c08467198c6ce04"
server_key_version: "encryptionkey@0"
I20260812 02:42:03.344980 1536 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786502156971023-7698-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786502156971023-7698-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786502156971023-7698-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:42:03.380091 1536 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:42:03.384851 1536 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:42:03.385143 1536 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:42:03.405720 1536 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.190:40183
I20260812 02:42:03.405743 1617 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.190:40183 every 8 connection(s)
I20260812 02:42:03.407505 1536 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
I20260812 02:42:03.411661 1618 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:42:03.414345 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 1536
I20260812 02:42:03.414579 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786502156971023-7698-0/minicluster-data/master-0/wal/instance
I20260812 02:42:03.415047 7698 external_mini_cluster.cc:1428] Setting key d9bc38890756c764f8c11ad0352ff865
I20260812 02:42:03.425556 1618 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 609672c317804723befcd0cf7c89114d: Bootstrap starting.
I20260812 02:42:03.428996 1618 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 609672c317804723befcd0cf7c89114d: Neither blocks nor log segments found. Creating new log.
I20260812 02:42:03.430260 1618 log.cc:824] T 00000000000000000000000000000000 P 609672c317804723befcd0cf7c89114d: Log is configured to *not* fsync() on all Append() calls
I20260812 02:42:03.431288 1621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:03.417030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49177 (local address 127.7.132.190:40183)
0812 02:42:03.417965 (+ 935us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:03.417983 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:03.418053 (+ 70us) server_negotiation.cc:407] Connection header received
0812 02:42:03.420492 (+ 2439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:03.420531 (+ 39us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:03.421099 (+ 568us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:03.421580 (+ 481us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:03.422198 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:03.424352 (+ 2154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:03.425848 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:03.426647 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:03.426860 (+ 213us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:03.428440 (+ 1580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:42:03.428449 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:42:03.428466 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:42:03.429072 (+ 606us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:42:03.429216 (+ 144us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:42:03.429265 (+ 49us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:42:03.429490 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:03.429815 (+ 325us) server_negotiation.cc:299] Negotiation successful
0812 02:42:03.430018 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":691,"thread_start_us":435,"threads_started":1}
I20260812 02:42:03.433831 1618 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 609672c317804723befcd0cf7c89114d: No bootstrap required, opened a new log
I20260812 02:42:03.435765 7698 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 02:42:03.446616 1618 raft_consensus.cc:348] T 00000000000000000000000000000000 P 609672c317804723befcd0cf7c89114d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "609672c317804723befcd0cf7c89114d" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 40183 } }
I20260812 02:42:03.447319 1618 raft_consensus.cc:374] T 00000000000000000000000000000000 P 609672c317804723befcd0cf7c89114d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:42:03.447467 1618 raft_consensus.cc:729] T 00000000000000000000000000000000 P 609672c317804723befcd0cf7c89114d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 609672c317804723befcd0cf7c89114d, State: Initialized, Role: FOLLOWER
I20260812 02:42:03.448331 1618 consensus_queue.cc:260] T 00000000000000000000000000000000 P 609672c317804723befcd0cf7c89114d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "609672c317804723befcd0cf7c89114d" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 40183 } }
I20260812 02:42:03.448689 1618 raft_consensus.cc:388] T 00000000000000000000000000000000 P 609672c317804723befcd0cf7c89114d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 02:42:03.448864 1618 raft_consensus.cc:482] T 00000000000000000000000000000000 P 609672c317804723befcd0cf7c89114d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 02:42:03.449218 1618 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 609672c317804723befcd0cf7c89114d [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:42:03.449965 1621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:03.442185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43435 (local address 127.7.132.190:40183)
0812 02:42:03.442437 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:03.442446 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:03.443018 (+ 572us) server_negotiation.cc:407] Connection header received
0812 02:42:03.443236 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:03.443240 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:03.443309 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:03.443397 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:03.444111 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:03.444904 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:03.446997 (+ 2093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:03.447585 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:03.447723 (+ 138us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:03.448981 (+ 1258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:42:03.448986 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:42:03.448989 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:42:03.449006 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:42:03.449113 (+ 107us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:42:03.449140 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:42:03.449246 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:03.449561 (+ 315us) server_negotiation.cc:299] Negotiation successful
0812 02:42:03.449613 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260812 02:42:03.452608 1618 raft_consensus.cc:504] T 00000000000000000000000000000000 P 609672c317804723befcd0cf7c89114d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "609672c317804723befcd0cf7c89114d" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 40183 } }
I20260812 02:42:03.453224 1618 leader_election.cc:302] T 00000000000000000000000000000000 P 609672c317804723befcd0cf7c89114d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 609672c317804723befcd0cf7c89114d; no voters:
I20260812 02:42:03.454043 1618 leader_election.cc:288] T 00000000000000000000000000000000 P 609672c317804723befcd0cf7c89114d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 02:42:03.454176 1629 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 609672c317804723befcd0cf7c89114d [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:42:03.454901 1629 raft_consensus.cc:686] T 00000000000000000000000000000000 P 609672c317804723befcd0cf7c89114d [term 1 LEADER]: Becoming Leader. State: Replica: 609672c317804723befcd0cf7c89114d, State: Running, Role: LEADER
I20260812 02:42:03.455513 1629 consensus_queue.cc:237] T 00000000000000000000000000000000 P 609672c317804723befcd0cf7c89114d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "609672c317804723befcd0cf7c89114d" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 40183 } }
I20260812 02:42:03.456670 1618 sys_catalog.cc:564] T 00000000000000000000000000000000 P 609672c317804723befcd0cf7c89114d [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:42:03.458705 1631 sys_catalog.cc:455] T 00000000000000000000000000000000 P 609672c317804723befcd0cf7c89114d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "609672c317804723befcd0cf7c89114d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "609672c317804723befcd0cf7c89114d" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 40183 } } }
I20260812 02:42:03.459086 1631 sys_catalog.cc:458] T 00000000000000000000000000000000 P 609672c317804723befcd0cf7c89114d [sys.catalog]: This master's current role is: LEADER
I20260812 02:42:03.461402 1637 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:42:03.461560 1634 sys_catalog.cc:455] T 00000000000000000000000000000000 P 609672c317804723befcd0cf7c89114d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 609672c317804723befcd0cf7c89114d. Latest consensus state: current_term: 1 leader_uuid: "609672c317804723befcd0cf7c89114d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "609672c317804723befcd0cf7c89114d" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 40183 } } }
I20260812 02:42:03.461982 1634 sys_catalog.cc:458] T 00000000000000000000000000000000 P 609672c317804723befcd0cf7c89114d [sys.catalog]: This master's current role is: LEADER
I20260812 02:42:03.466414 1637 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 02:42:03.475443 1637 catalog_manager.cc:1347] Generated new cluster ID: a17458ced68443b69cc8368a29493486
I20260812 02:42:03.475569 1637 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 02:42:03.496716 1637 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 02:42:03.498055 1637 catalog_manager.cc:1504] Loading token signing keys...
I20260812 02:42:03.506474 1637 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 609672c317804723befcd0cf7c89114d: Generated new TSK 0
I20260812 02:42:03.507505 1637 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 02:42:03.515439 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 1536
W20260812 02:42:03.516711 1611 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:49177
W20260812 02:42:03.517503 7698 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 1536: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 02:42:03.517668 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 1536
W20260812 02:42:03.540743 1517 connection.cc:535] client connection to 127.7.132.190:40183 recv error: Network error: failed to read from TLS socket (remote: 127.7.132.190:40183): Connection reset by peer (error 104)
2026-08-12T02:42:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (2174 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786502156971023-7698-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 02:42:03 dist-test-slave-mn4r krb5kdc[1660](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 02:42:03 dist-test-slave-mn4r krb5kdc[1660](info): set up 2 sockets
Aug 12 02:42:03 dist-test-slave-mn4r krb5kdc[1660](info): commencing operation
krb5kdc: starting...
W20260812 02:42:04.777750 7698 mini_kdc.cc:121] Time spent starting KDC: real 1.217s user 0.000s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786502156971023-7698-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786502156971023-7698-0/krb5kdc/test-user.keytab.
Aug 12 02:42:04 dist-test-slave-mn4r krb5kdc[1660](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502524, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T02:42:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T02:42:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.132.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
I20260812 02:42:05.103929 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786502156971023-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786502156971023-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786502156971023-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:39415
--webserver_interface=127.7.132.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:46511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.132.190:39415
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:42:05.462782 1676 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:42:05.463157 1676 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:42:05.463325 1676 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:42:05.470695 1676 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:42:05.470856 1676 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:42:05.470983 1676 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:42:05.471103 1676 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:42:05.471182 1676 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:42:05.481263 1676 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:46511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786502156971023-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786502156971023-7698-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.132.190:39415
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:39415
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.132.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786502156971023-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:42:05.482684 1676 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:42:05.484135 1676 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:42:05.496999 1682 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:42:05.497099 1683 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:42:05.497129 1685 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:42:05.497819 1676 server_base.cc:1034] running on GCE node
I20260812 02:42:05.498695 1676 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:42:05.500111 1676 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:42:05.501363 1676 hybrid_clock.cc:648] HybridClock initialized: now 1786502525501311 us; error 46 us; skew 500 ppm
I20260812 02:42:05.501753 1676 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:42:05 dist-test-slave-mn4r krb5kdc[1660](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502525, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:42:05.510406 1676 init.cc:377] Logged in from keytab as kudu/127.7.132.190@KRBTEST.COM (short username kudu)
I20260812 02:42:05.512050 1676 webserver.cc:460] Webserver started at http://127.7.132.190:45919/ using document root <none> and password file <none>
I20260812 02:42:05.512768 1676 fs_manager.cc:359] Metadata directory not provided
I20260812 02:42:05.512888 1676 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:42:05.513176 1676 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:42:05.516520 1676 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786502156971023-7698-0/minicluster-data/master-0/data/instance:
uuid: "0f7c1311323e45e9aa35ce0e8f629d47"
format_stamp: "Formatted at 2026-08-12 02:42:05 on dist-test-slave-mn4r"
server_key: "8c33e5b4abc64c3930a8c10c7976b2cf"
server_key_iv: "1572c6bb03e90034a2349ecbc0d581e4"
server_key_version: "encryptionkey@0"
I20260812 02:42:05.517315 1676 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786502156971023-7698-0/minicluster-data/master-0/wal/instance:
uuid: "0f7c1311323e45e9aa35ce0e8f629d47"
format_stamp: "Formatted at 2026-08-12 02:42:05 on dist-test-slave-mn4r"
server_key: "8c33e5b4abc64c3930a8c10c7976b2cf"
server_key_iv: "1572c6bb03e90034a2349ecbc0d581e4"
server_key_version: "encryptionkey@0"
I20260812 02:42:05.522744 1676 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260812 02:42:05.526448 1692 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:42:05.527977 1676 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 02:42:05.528160 1676 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786502156971023-7698-0/minicluster-data/master-0/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786502156971023-7698-0/minicluster-data/master-0/wal
uuid: "0f7c1311323e45e9aa35ce0e8f629d47"
format_stamp: "Formatted at 2026-08-12 02:42:05 on dist-test-slave-mn4r"
server_key: "8c33e5b4abc64c3930a8c10c7976b2cf"
server_key_iv: "1572c6bb03e90034a2349ecbc0d581e4"
server_key_version: "encryptionkey@0"
I20260812 02:42:05.528378 1676 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786502156971023-7698-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786502156971023-7698-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786502156971023-7698-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:42:05.534377 1676 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:42:05.538431 1676 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:42:05.538743 1676 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:42:05.557680 1676 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.190:39415
I20260812 02:42:05.557719 1749 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.190:39415 every 8 connection(s)
I20260812 02:42:05.559195 1676 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
I20260812 02:42:05.563015 1751 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:42:05.567764 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 1676
I20260812 02:42:05.567942 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786502156971023-7698-0/minicluster-data/master-0/wal/instance
I20260812 02:42:05.568408 7698 external_mini_cluster.cc:1428] Setting key a619cf9e81ec66131a82eb26535c98e5
I20260812 02:42:05.573993 1751 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f7c1311323e45e9aa35ce0e8f629d47: Bootstrap starting.
I20260812 02:42:05.576192 1755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:05.570235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50175 (local address 127.7.132.190:39415)
0812 02:42:05.571063 (+ 828us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:05.571074 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:05.571117 (+ 43us) server_negotiation.cc:407] Connection header received
0812 02:42:05.572823 (+ 1706us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:05.572844 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:05.573171 (+ 327us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:05.573474 (+ 303us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:05.574637 (+ 1163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:42:05.574643 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:42:05.574656 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:42:05.574671 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:42:05.574729 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:42:05.574753 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:42:05.574988 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:05.575118 (+ 130us) server_negotiation.cc:299] Negotiation successful
0812 02:42:05.575297 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":616,"thread_start_us":399,"threads_started":1}
I20260812 02:42:05.577572 1751 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0f7c1311323e45e9aa35ce0e8f629d47: Neither blocks nor log segments found. Creating new log.
I20260812 02:42:05.578850 1751 log.cc:824] T 00000000000000000000000000000000 P 0f7c1311323e45e9aa35ce0e8f629d47: Log is configured to *not* fsync() on all Append() calls
I20260812 02:42:05.580159 7698 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 02:42:05.581724 1751 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f7c1311323e45e9aa35ce0e8f629d47: No bootstrap required, opened a new log
I20260812 02:42:05.587328 1755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:05.584677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57855 (local address 127.7.132.190:39415)
0812 02:42:05.584828 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:05.584832 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:05.585371 (+ 539us) server_negotiation.cc:407] Connection header received
0812 02:42:05.585542 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:05.585545 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:05.585588 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:05.585682 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:05.586652 (+ 970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:42:05.586656 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:42:05.586659 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:42:05.586673 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:42:05.586713 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:42:05.586730 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:42:05.586817 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:05.586968 (+ 151us) server_negotiation.cc:299] Negotiation successful
0812 02:42:05.587016 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 02:42:05.590718 1751 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0f7c1311323e45e9aa35ce0e8f629d47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f7c1311323e45e9aa35ce0e8f629d47" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 39415 } }
I20260812 02:42:05.591089 1751 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0f7c1311323e45e9aa35ce0e8f629d47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:42:05.591195 1751 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0f7c1311323e45e9aa35ce0e8f629d47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f7c1311323e45e9aa35ce0e8f629d47, State: Initialized, Role: FOLLOWER
I20260812 02:42:05.591780 1751 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0f7c1311323e45e9aa35ce0e8f629d47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f7c1311323e45e9aa35ce0e8f629d47" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 39415 } }
I20260812 02:42:05.592029 1751 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0f7c1311323e45e9aa35ce0e8f629d47 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 02:42:05.592173 1751 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0f7c1311323e45e9aa35ce0e8f629d47 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 02:42:05.592356 1751 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0f7c1311323e45e9aa35ce0e8f629d47 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:42:05.594805 1751 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0f7c1311323e45e9aa35ce0e8f629d47 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f7c1311323e45e9aa35ce0e8f629d47" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 39415 } }
I20260812 02:42:05.595252 1751 leader_election.cc:302] T 00000000000000000000000000000000 P 0f7c1311323e45e9aa35ce0e8f629d47 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0f7c1311323e45e9aa35ce0e8f629d47; no voters:
I20260812 02:42:05.595963 1751 leader_election.cc:288] T 00000000000000000000000000000000 P 0f7c1311323e45e9aa35ce0e8f629d47 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 02:42:05.596086 1767 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0f7c1311323e45e9aa35ce0e8f629d47 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:42:05.596750 1767 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0f7c1311323e45e9aa35ce0e8f629d47 [term 1 LEADER]: Becoming Leader. State: Replica: 0f7c1311323e45e9aa35ce0e8f629d47, State: Running, Role: LEADER
I20260812 02:42:05.597354 1767 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0f7c1311323e45e9aa35ce0e8f629d47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f7c1311323e45e9aa35ce0e8f629d47" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 39415 } }
I20260812 02:42:05.598318 1751 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0f7c1311323e45e9aa35ce0e8f629d47 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:42:05.600267 1768 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f7c1311323e45e9aa35ce0e8f629d47 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0f7c1311323e45e9aa35ce0e8f629d47" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f7c1311323e45e9aa35ce0e8f629d47" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 39415 } } }
I20260812 02:42:05.600656 1768 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f7c1311323e45e9aa35ce0e8f629d47 [sys.catalog]: This master's current role is: LEADER
I20260812 02:42:05.602936 1769 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f7c1311323e45e9aa35ce0e8f629d47 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0f7c1311323e45e9aa35ce0e8f629d47. Latest consensus state: current_term: 1 leader_uuid: "0f7c1311323e45e9aa35ce0e8f629d47" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f7c1311323e45e9aa35ce0e8f629d47" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 39415 } } }
I20260812 02:42:05.603140 1773 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:42:05.603268 1769 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f7c1311323e45e9aa35ce0e8f629d47 [sys.catalog]: This master's current role is: LEADER
I20260812 02:42:05.609612 1773 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 02:42:05.620100 1773 catalog_manager.cc:1347] Generated new cluster ID: 5ef0a6756c1e4acdacc531f652b3e052
I20260812 02:42:05.620215 1773 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 02:42:05.636961 1773 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 02:42:05.637455 1773 catalog_manager.cc:1504] Loading token signing keys...
I20260812 02:42:05.648998 1773 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0f7c1311323e45e9aa35ce0e8f629d47: Generated new TSK 0
I20260812 02:42:05.650053 1773 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 02:42:05.662703 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 1676
W20260812 02:42:05.663941 1745 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:50175
W20260812 02:42:05.664706 7698 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 1676: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 02:42:05.664865 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 1676
2026-08-12T02:42:05Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (2148 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786502156971023-7698-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 02:42:05 dist-test-slave-mn4r krb5kdc[1789](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 02:42:05 dist-test-slave-mn4r krb5kdc[1789](info): set up 2 sockets
Aug 12 02:42:05 dist-test-slave-mn4r krb5kdc[1789](info): commencing operation
krb5kdc: starting...
W20260812 02:42:06.900049 7698 mini_kdc.cc:121] Time spent starting KDC: real 1.191s user 0.000s sys 0.048s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786502156971023-7698-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786502156971023-7698-0/krb5kdc/test-user.keytab.
Aug 12 02:42:07 dist-test-slave-mn4r krb5kdc[1789](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502527, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T02:42:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T02:42:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.132.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
I20260812 02:42:07.228823 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786502156971023-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786502156971023-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786502156971023-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:41355
--webserver_interface=127.7.132.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:35571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.132.190:41355
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:42:07.582230 1802 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:42:07.582530 1802 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:42:07.582693 1802 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:42:07.590231 1802 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:42:07.590332 1802 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:42:07.590461 1802 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:42:07.590564 1802 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:42:07.590651 1802 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:42:07.601029 1802 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:35571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786502156971023-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786502156971023-7698-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.132.190:41355
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:41355
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.132.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786502156971023-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:42:07.602434 1802 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:42:07.603785 1802 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:42:07.615801 1810 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:42:07.615854 1814 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:42:07.615756 1811 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:42:07.617524 1802 server_base.cc:1034] running on GCE node
I20260812 02:42:07.618389 1802 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:42:07.619884 1802 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:42:07.621132 1802 hybrid_clock.cc:648] HybridClock initialized: now 1786502527621090 us; error 31 us; skew 500 ppm
I20260812 02:42:07.621512 1802 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:42:07 dist-test-slave-mn4r krb5kdc[1789](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502527, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:42:07.624843 1802 init.cc:377] Logged in from keytab as kudu/127.7.132.190@KRBTEST.COM (short username kudu)
I20260812 02:42:07.626381 1802 webserver.cc:460] Webserver started at http://127.7.132.190:37249/ using document root <none> and password file <none>
I20260812 02:42:07.627053 1802 fs_manager.cc:359] Metadata directory not provided
I20260812 02:42:07.627157 1802 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:42:07.627410 1802 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:42:07.630530 1802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786502156971023-7698-0/minicluster-data/master-0/data/instance:
uuid: "fead0eed3e5543b99c92c29175e6c2a8"
format_stamp: "Formatted at 2026-08-12 02:42:07 on dist-test-slave-mn4r"
server_key: "0b43126b488f742e9086457bf115dd6c"
server_key_iv: "06a1419547afc0f4315d6b4b0d69c395"
server_key_version: "encryptionkey@0"
I20260812 02:42:07.631254 1802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786502156971023-7698-0/minicluster-data/master-0/wal/instance:
uuid: "fead0eed3e5543b99c92c29175e6c2a8"
format_stamp: "Formatted at 2026-08-12 02:42:07 on dist-test-slave-mn4r"
server_key: "0b43126b488f742e9086457bf115dd6c"
server_key_iv: "06a1419547afc0f4315d6b4b0d69c395"
server_key_version: "encryptionkey@0"
I20260812 02:42:07.636477 1802 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.005s
I20260812 02:42:07.639995 1822 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:42:07.641453 1802 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260812 02:42:07.641640 1802 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786502156971023-7698-0/minicluster-data/master-0/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786502156971023-7698-0/minicluster-data/master-0/wal
uuid: "fead0eed3e5543b99c92c29175e6c2a8"
format_stamp: "Formatted at 2026-08-12 02:42:07 on dist-test-slave-mn4r"
server_key: "0b43126b488f742e9086457bf115dd6c"
server_key_iv: "06a1419547afc0f4315d6b4b0d69c395"
server_key_version: "encryptionkey@0"
I20260812 02:42:07.641826 1802 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786502156971023-7698-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786502156971023-7698-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786502156971023-7698-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:42:07.666998 1802 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:42:07.671252 1802 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:42:07.671558 1802 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:42:07.690698 1802 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.190:41355
I20260812 02:42:07.690744 1890 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.190:41355 every 8 connection(s)
I20260812 02:42:07.692102 1802 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
I20260812 02:42:07.696241 1891 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:42:07.701699 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 1802
I20260812 02:42:07.701922 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786502156971023-7698-0/minicluster-data/master-0/wal/instance
I20260812 02:42:07.702585 7698 external_mini_cluster.cc:1428] Setting key 2169384162a55e04baac6f51db3ff746
I20260812 02:42:07.709147 1891 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fead0eed3e5543b99c92c29175e6c2a8: Bootstrap starting.
I20260812 02:42:07.712530 1891 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fead0eed3e5543b99c92c29175e6c2a8: Neither blocks nor log segments found. Creating new log.
I20260812 02:42:07.713739 1891 log.cc:824] T 00000000000000000000000000000000 P fead0eed3e5543b99c92c29175e6c2a8: Log is configured to *not* fsync() on all Append() calls
I20260812 02:42:07.716831 1891 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fead0eed3e5543b99c92c29175e6c2a8: No bootstrap required, opened a new log
Aug 12 02:42:07 dist-test-slave-mn4r krb5kdc[1789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502527, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 02:42:07.728606 1894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:07.704700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37239 (local address 127.7.132.190:41355)
0812 02:42:07.705588 (+ 888us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:07.705601 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:07.705641 (+ 40us) server_negotiation.cc:407] Connection header received
0812 02:42:07.707539 (+ 1898us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:07.707563 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:07.707945 (+ 382us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:07.708331 (+ 386us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:07.709862 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:07.711280 (+ 1418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:07.712633 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:07.713378 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:07.713603 (+ 225us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:07.718529 (+ 4926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:42:07.718542 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:42:07.718572 (+ 30us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:42:07.718622 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:42:07.723421 (+ 4799us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:07.724140 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:07.724156 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:07.724165 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:07.724400 (+ 235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:07.724782 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:07.724790 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:07.724797 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:07.725629 (+ 832us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:42:07.725976 (+ 347us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:07.726483 (+ 507us) server_negotiation.cc:299] Negotiation successful
0812 02:42:07.727014 (+ 531us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":641,"thread_start_us":407,"threads_started":1}
I20260812 02:42:07.729923 1891 raft_consensus.cc:348] T 00000000000000000000000000000000 P fead0eed3e5543b99c92c29175e6c2a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fead0eed3e5543b99c92c29175e6c2a8" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 41355 } }
I20260812 02:42:07.730505 1891 raft_consensus.cc:374] T 00000000000000000000000000000000 P fead0eed3e5543b99c92c29175e6c2a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:42:07.730645 1891 raft_consensus.cc:729] T 00000000000000000000000000000000 P fead0eed3e5543b99c92c29175e6c2a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fead0eed3e5543b99c92c29175e6c2a8, State: Initialized, Role: FOLLOWER
I20260812 02:42:07.731427 1891 consensus_queue.cc:260] T 00000000000000000000000000000000 P fead0eed3e5543b99c92c29175e6c2a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fead0eed3e5543b99c92c29175e6c2a8" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 41355 } }
I20260812 02:42:07.731762 1891 raft_consensus.cc:388] T 00000000000000000000000000000000 P fead0eed3e5543b99c92c29175e6c2a8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 02:42:07.731927 1891 raft_consensus.cc:482] T 00000000000000000000000000000000 P fead0eed3e5543b99c92c29175e6c2a8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 02:42:07.732158 1891 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fead0eed3e5543b99c92c29175e6c2a8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:42:07.732759 7698 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 02:42:07.734774 1891 raft_consensus.cc:504] T 00000000000000000000000000000000 P fead0eed3e5543b99c92c29175e6c2a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fead0eed3e5543b99c92c29175e6c2a8" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 41355 } }
I20260812 02:42:07.735339 1891 leader_election.cc:302] T 00000000000000000000000000000000 P fead0eed3e5543b99c92c29175e6c2a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fead0eed3e5543b99c92c29175e6c2a8; no voters:
I20260812 02:42:07.736119 1891 leader_election.cc:288] T 00000000000000000000000000000000 P fead0eed3e5543b99c92c29175e6c2a8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 02:42:07.736212 1900 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fead0eed3e5543b99c92c29175e6c2a8 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:42:07.737882 1900 raft_consensus.cc:686] T 00000000000000000000000000000000 P fead0eed3e5543b99c92c29175e6c2a8 [term 1 LEADER]: Becoming Leader. State: Replica: fead0eed3e5543b99c92c29175e6c2a8, State: Running, Role: LEADER
I20260812 02:42:07.738533 1900 consensus_queue.cc:237] T 00000000000000000000000000000000 P fead0eed3e5543b99c92c29175e6c2a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fead0eed3e5543b99c92c29175e6c2a8" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 41355 } }
I20260812 02:42:07.738767 1891 sys_catalog.cc:564] T 00000000000000000000000000000000 P fead0eed3e5543b99c92c29175e6c2a8 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:42:07.742189 1905 sys_catalog.cc:455] T 00000000000000000000000000000000 P fead0eed3e5543b99c92c29175e6c2a8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fead0eed3e5543b99c92c29175e6c2a8. Latest consensus state: current_term: 1 leader_uuid: "fead0eed3e5543b99c92c29175e6c2a8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fead0eed3e5543b99c92c29175e6c2a8" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 41355 } } }
I20260812 02:42:07.742561 1905 sys_catalog.cc:458] T 00000000000000000000000000000000 P fead0eed3e5543b99c92c29175e6c2a8 [sys.catalog]: This master's current role is: LEADER
I20260812 02:42:07.746655 1906 sys_catalog.cc:455] T 00000000000000000000000000000000 P fead0eed3e5543b99c92c29175e6c2a8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fead0eed3e5543b99c92c29175e6c2a8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fead0eed3e5543b99c92c29175e6c2a8" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 41355 } } }
I20260812 02:42:07.747093 1912 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:42:07.747090 1906 sys_catalog.cc:458] T 00000000000000000000000000000000 P fead0eed3e5543b99c92c29175e6c2a8 [sys.catalog]: This master's current role is: LEADER
I20260812 02:42:07.757730 1912 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 02:42:07.759927 1894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:07.737298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38613 (local address 127.7.132.190:41355)
0812 02:42:07.737451 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:07.737459 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:07.738115 (+ 656us) server_negotiation.cc:407] Connection header received
0812 02:42:07.738403 (+ 288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:07.738417 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:07.738500 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:07.738627 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:07.740597 (+ 1970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:07.742403 (+ 1806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:07.744948 (+ 2545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:07.745870 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:07.746031 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:07.749824 (+ 3793us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:42:07.749835 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:42:07.749843 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:42:07.749899 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:42:07.755226 (+ 5327us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:07.756229 (+ 1003us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:07.756240 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:07.756244 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:07.756401 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:07.756940 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:07.756952 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:07.756957 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:07.757461 (+ 504us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:42:07.757762 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:07.759230 (+ 1468us) server_negotiation.cc:299] Negotiation successful
0812 02:42:07.759491 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 02:42:07.774106 1912 catalog_manager.cc:1347] Generated new cluster ID: 79fb6e37a3d243f5afc3a2023c05452c
I20260812 02:42:07.774241 1912 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 02:42:07.805617 1912 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 02:42:07.807785 1912 catalog_manager.cc:1504] Loading token signing keys...
I20260812 02:42:07.817435 1912 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fead0eed3e5543b99c92c29175e6c2a8: Generated new TSK 0
I20260812 02:42:07.818750 1912 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 02:42:07.828791 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 1802
I20260812 02:42:07.830250 1885 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:42:07.923337 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 1802
2026-08-12T02:42:07Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (2254 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786502156971023-7698-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 02:42:08 dist-test-slave-mn4r krb5kdc[1933](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 02:42:08 dist-test-slave-mn4r krb5kdc[1933](info): set up 2 sockets
Aug 12 02:42:08 dist-test-slave-mn4r krb5kdc[1933](info): commencing operation
krb5kdc: starting...
W20260812 02:42:09.158630 7698 mini_kdc.cc:121] Time spent starting KDC: real 1.195s user 0.011s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786502156971023-7698-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786502156971023-7698-0/krb5kdc/test-user.keytab.
Aug 12 02:42:09 dist-test-slave-mn4r krb5kdc[1933](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502529, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T02:42:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T02:42:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.132.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
I20260812 02:42:09.507325 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786502156971023-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786502156971023-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786502156971023-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:45459
--webserver_interface=127.7.132.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:41575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.132.190:45459
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:42:09.875088 1949 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:42:09.875489 1949 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:42:09.875666 1949 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:42:09.883038 1949 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:42:09.883212 1949 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:42:09.883327 1949 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:42:09.883440 1949 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:42:09.883530 1949 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:42:09.893672 1949 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:41575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786502156971023-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786502156971023-7698-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.132.190:45459
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:45459
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.132.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786502156971023-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:42:09.895155 1949 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:42:09.896627 1949 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:42:09.909538 1954 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:42:09.909590 1957 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:42:09.909577 1955 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:42:09.911033 1949 server_base.cc:1034] running on GCE node
I20260812 02:42:09.912005 1949 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:42:09.913551 1949 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:42:09.914801 1949 hybrid_clock.cc:648] HybridClock initialized: now 1786502529914759 us; error 56 us; skew 500 ppm
I20260812 02:42:09.915256 1949 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:42:09 dist-test-slave-mn4r krb5kdc[1933](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502529, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:42:09.918764 1949 init.cc:377] Logged in from keytab as kudu/127.7.132.190@KRBTEST.COM (short username kudu)
I20260812 02:42:09.920353 1949 webserver.cc:460] Webserver started at http://127.7.132.190:35131/ using document root <none> and password file <none>
I20260812 02:42:09.921053 1949 fs_manager.cc:359] Metadata directory not provided
I20260812 02:42:09.921219 1949 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:42:09.921550 1949 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:42:09.924868 1949 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786502156971023-7698-0/minicluster-data/master-0/data/instance:
uuid: "a8ca05ef18bd472eb84dc507492e2330"
format_stamp: "Formatted at 2026-08-12 02:42:09 on dist-test-slave-mn4r"
server_key: "e6f9afb8ca645e9505873ec4f6427d96"
server_key_iv: "e8ece8b85ee9a6a00306879197189f36"
server_key_version: "encryptionkey@0"
I20260812 02:42:09.925644 1949 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786502156971023-7698-0/minicluster-data/master-0/wal/instance:
uuid: "a8ca05ef18bd472eb84dc507492e2330"
format_stamp: "Formatted at 2026-08-12 02:42:09 on dist-test-slave-mn4r"
server_key: "e6f9afb8ca645e9505873ec4f6427d96"
server_key_iv: "e8ece8b85ee9a6a00306879197189f36"
server_key_version: "encryptionkey@0"
I20260812 02:42:09.931181 1949 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260812 02:42:09.934940 1965 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:42:09.936349 1949 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 02:42:09.936553 1949 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786502156971023-7698-0/minicluster-data/master-0/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786502156971023-7698-0/minicluster-data/master-0/wal
uuid: "a8ca05ef18bd472eb84dc507492e2330"
format_stamp: "Formatted at 2026-08-12 02:42:09 on dist-test-slave-mn4r"
server_key: "e6f9afb8ca645e9505873ec4f6427d96"
server_key_iv: "e8ece8b85ee9a6a00306879197189f36"
server_key_version: "encryptionkey@0"
I20260812 02:42:09.936767 1949 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786502156971023-7698-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786502156971023-7698-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786502156971023-7698-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:42:09.950979 1949 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:42:09.955325 1949 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:42:09.955628 1949 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:42:09.975060 1949 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.190:45459
I20260812 02:42:09.975085 2034 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.190:45459 every 8 connection(s)
I20260812 02:42:09.976527 1949 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
I20260812 02:42:09.980468 2035 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:42:09.982888 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 1949
I20260812 02:42:09.983095 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786502156971023-7698-0/minicluster-data/master-0/wal/instance
I20260812 02:42:09.983659 7698 external_mini_cluster.cc:1428] Setting key ccd38592e04e74bf2fad14eedc6857bc
I20260812 02:42:09.993597 2035 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a8ca05ef18bd472eb84dc507492e2330: Bootstrap starting.
I20260812 02:42:09.997502 2035 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a8ca05ef18bd472eb84dc507492e2330: Neither blocks nor log segments found. Creating new log.
I20260812 02:42:09.997835 2038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:09.985605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60287 (local address 127.7.132.190:45459)
0812 02:42:09.986478 (+ 873us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:09.986490 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:09.986544 (+ 54us) server_negotiation.cc:407] Connection header received
0812 02:42:09.988273 (+ 1729us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:09.988297 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:09.988714 (+ 417us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:09.989043 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:09.989706 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:09.991198 (+ 1492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:09.993016 (+ 1818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:09.993734 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:09.993919 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:09.995316 (+ 1397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:42:09.995324 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:42:09.995342 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:42:09.995927 (+ 585us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:42:09.996013 (+ 86us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:42:09.996037 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:42:09.996128 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:09.996372 (+ 244us) server_negotiation.cc:299] Negotiation successful
0812 02:42:09.996689 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":654,"thread_start_us":426,"threads_started":1}
I20260812 02:42:09.998970 2035 log.cc:824] T 00000000000000000000000000000000 P a8ca05ef18bd472eb84dc507492e2330: Log is configured to *not* fsync() on all Append() calls
I20260812 02:42:10.002197 7698 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 02:42:10.002408 2035 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a8ca05ef18bd472eb84dc507492e2330: No bootstrap required, opened a new log
I20260812 02:42:10.011960 2035 raft_consensus.cc:348] T 00000000000000000000000000000000 P a8ca05ef18bd472eb84dc507492e2330 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8ca05ef18bd472eb84dc507492e2330" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 45459 } }
I20260812 02:42:10.012550 2035 raft_consensus.cc:374] T 00000000000000000000000000000000 P a8ca05ef18bd472eb84dc507492e2330 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:42:10.012689 2035 raft_consensus.cc:729] T 00000000000000000000000000000000 P a8ca05ef18bd472eb84dc507492e2330 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8ca05ef18bd472eb84dc507492e2330, State: Initialized, Role: FOLLOWER
I20260812 02:42:10.013511 2035 consensus_queue.cc:260] T 00000000000000000000000000000000 P a8ca05ef18bd472eb84dc507492e2330 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8ca05ef18bd472eb84dc507492e2330" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 45459 } }
I20260812 02:42:10.013863 2035 raft_consensus.cc:388] T 00000000000000000000000000000000 P a8ca05ef18bd472eb84dc507492e2330 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 02:42:10.014032 2035 raft_consensus.cc:482] T 00000000000000000000000000000000 P a8ca05ef18bd472eb84dc507492e2330 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 02:42:10.014351 2035 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a8ca05ef18bd472eb84dc507492e2330 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:42:10.014381 2038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:10.006852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54341 (local address 127.7.132.190:45459)
0812 02:42:10.007086 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:10.007094 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:10.007709 (+ 615us) server_negotiation.cc:407] Connection header received
0812 02:42:10.007931 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:10.007938 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:10.008010 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:10.008108 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:10.008786 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:10.009540 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:10.011596 (+ 2056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:10.012038 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:10.012138 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:10.013406 (+ 1268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:42:10.013411 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:42:10.013414 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:42:10.013435 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:42:10.013504 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:42:10.013521 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:42:10.013602 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:10.013811 (+ 209us) server_negotiation.cc:299] Negotiation successful
0812 02:42:10.013925 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 02:42:10.017227 2035 raft_consensus.cc:504] T 00000000000000000000000000000000 P a8ca05ef18bd472eb84dc507492e2330 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8ca05ef18bd472eb84dc507492e2330" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 45459 } }
I20260812 02:42:10.017800 2035 leader_election.cc:302] T 00000000000000000000000000000000 P a8ca05ef18bd472eb84dc507492e2330 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a8ca05ef18bd472eb84dc507492e2330; no voters:
I20260812 02:42:10.018467 2035 leader_election.cc:288] T 00000000000000000000000000000000 P a8ca05ef18bd472eb84dc507492e2330 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 02:42:10.018612 2047 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a8ca05ef18bd472eb84dc507492e2330 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:42:10.019371 2047 raft_consensus.cc:686] T 00000000000000000000000000000000 P a8ca05ef18bd472eb84dc507492e2330 [term 1 LEADER]: Becoming Leader. State: Replica: a8ca05ef18bd472eb84dc507492e2330, State: Running, Role: LEADER
I20260812 02:42:10.020023 2047 consensus_queue.cc:237] T 00000000000000000000000000000000 P a8ca05ef18bd472eb84dc507492e2330 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8ca05ef18bd472eb84dc507492e2330" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 45459 } }
I20260812 02:42:10.020776 2035 sys_catalog.cc:564] T 00000000000000000000000000000000 P a8ca05ef18bd472eb84dc507492e2330 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:42:10.023074 2048 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8ca05ef18bd472eb84dc507492e2330 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a8ca05ef18bd472eb84dc507492e2330" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8ca05ef18bd472eb84dc507492e2330" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 45459 } } }
I20260812 02:42:10.023517 2048 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8ca05ef18bd472eb84dc507492e2330 [sys.catalog]: This master's current role is: LEADER
I20260812 02:42:10.024055 2049 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8ca05ef18bd472eb84dc507492e2330 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a8ca05ef18bd472eb84dc507492e2330. Latest consensus state: current_term: 1 leader_uuid: "a8ca05ef18bd472eb84dc507492e2330" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8ca05ef18bd472eb84dc507492e2330" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 45459 } } }
I20260812 02:42:10.024439 2049 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8ca05ef18bd472eb84dc507492e2330 [sys.catalog]: This master's current role is: LEADER
I20260812 02:42:10.026769 2058 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:42:10.031699 2058 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 02:42:10.041100 2058 catalog_manager.cc:1347] Generated new cluster ID: b5006c02736041eeaa66a5154d672446
I20260812 02:42:10.041616 2058 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 02:42:10.090375 2058 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 02:42:10.091925 2058 catalog_manager.cc:1504] Loading token signing keys...
I20260812 02:42:10.101281 2058 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a8ca05ef18bd472eb84dc507492e2330: Generated new TSK 0
I20260812 02:42:10.102519 2058 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 02:42:10.110888 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 1949
W20260812 02:42:10.112099 2028 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:60287
W20260812 02:42:10.112872 7698 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 1949: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 02:42:10.113046 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 1949
2026-08-12T02:42:10Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (2189 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786502156971023-7698-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 02:42:10 dist-test-slave-mn4r krb5kdc[2075](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 02:42:10 dist-test-slave-mn4r krb5kdc[2075](info): set up 2 sockets
Aug 12 02:42:10 dist-test-slave-mn4r krb5kdc[2075](info): commencing operation
krb5kdc: starting...
W20260812 02:42:11.366624 7698 mini_kdc.cc:121] Time spent starting KDC: real 1.214s user 0.003s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786502156971023-7698-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786502156971023-7698-0/krb5kdc/test-user.keytab.
Aug 12 02:42:11 dist-test-slave-mn4r krb5kdc[2075](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502531, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T02:42:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T02:42:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.132.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
I20260812 02:42:11.713119 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786502156971023-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786502156971023-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786502156971023-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:35399
--webserver_interface=127.7.132.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:42943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.132.190:35399
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:42:12.079098 2091 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:42:12.079471 2091 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:42:12.079654 2091 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:42:12.087133 2091 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:42:12.087334 2091 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:42:12.087455 2091 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:42:12.087569 2091 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:42:12.087657 2091 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:42:12.098008 2091 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:42943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786502156971023-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786502156971023-7698-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.132.190:35399
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:35399
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.132.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786502156971023-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:42:12.099468 2091 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:42:12.100924 2091 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:42:12.114046 2102 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:42:12.114156 2099 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:42:12.114123 2100 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:42:12.115410 2091 server_base.cc:1034] running on GCE node
I20260812 02:42:12.116349 2091 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:42:12.117903 2091 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:42:12.119180 2091 hybrid_clock.cc:648] HybridClock initialized: now 1786502532119123 us; error 62 us; skew 500 ppm
I20260812 02:42:12.119603 2091 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:42:12 dist-test-slave-mn4r krb5kdc[2075](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502532, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:42:12.128947 2091 init.cc:377] Logged in from keytab as kudu/127.7.132.190@KRBTEST.COM (short username kudu)
I20260812 02:42:12.130788 2091 webserver.cc:460] Webserver started at http://127.7.132.190:39763/ using document root <none> and password file <none>
I20260812 02:42:12.131585 2091 fs_manager.cc:359] Metadata directory not provided
I20260812 02:42:12.131740 2091 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:42:12.132085 2091 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:42:12.135423 2091 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786502156971023-7698-0/minicluster-data/master-0/data/instance:
uuid: "e16fd4169098469d914d621f0a221468"
format_stamp: "Formatted at 2026-08-12 02:42:12 on dist-test-slave-mn4r"
server_key: "c28a2ec17c96168b2d27aa31a4f33de8"
server_key_iv: "06fa0713d16724e616523c3d4d686ae9"
server_key_version: "encryptionkey@0"
I20260812 02:42:12.136276 2091 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786502156971023-7698-0/minicluster-data/master-0/wal/instance:
uuid: "e16fd4169098469d914d621f0a221468"
format_stamp: "Formatted at 2026-08-12 02:42:12 on dist-test-slave-mn4r"
server_key: "c28a2ec17c96168b2d27aa31a4f33de8"
server_key_iv: "06fa0713d16724e616523c3d4d686ae9"
server_key_version: "encryptionkey@0"
I20260812 02:42:12.141822 2091 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260812 02:42:12.145807 2109 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:42:12.147859 2091 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.003s
I20260812 02:42:12.148075 2091 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786502156971023-7698-0/minicluster-data/master-0/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786502156971023-7698-0/minicluster-data/master-0/wal
uuid: "e16fd4169098469d914d621f0a221468"
format_stamp: "Formatted at 2026-08-12 02:42:12 on dist-test-slave-mn4r"
server_key: "c28a2ec17c96168b2d27aa31a4f33de8"
server_key_iv: "06fa0713d16724e616523c3d4d686ae9"
server_key_version: "encryptionkey@0"
I20260812 02:42:12.148300 2091 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786502156971023-7698-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786502156971023-7698-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786502156971023-7698-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:42:12.154480 2091 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:42:12.158751 2091 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:42:12.159094 2091 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:42:12.179039 2179 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.190:35399 every 8 connection(s)
I20260812 02:42:12.179050 2091 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.190:35399
I20260812 02:42:12.180792 2091 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
I20260812 02:42:12.184970 2181 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:42:12.187111 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 2091
I20260812 02:42:12.187309 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786502156971023-7698-0/minicluster-data/master-0/wal/instance
I20260812 02:42:12.187762 7698 external_mini_cluster.cc:1428] Setting key e8a004eb56bc3ca1070d801b8ed917c2
I20260812 02:42:12.196615 2185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:12.189778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59977 (local address 127.7.132.190:35399)
0812 02:42:12.190668 (+ 890us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:12.190692 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:12.190755 (+ 63us) server_negotiation.cc:407] Connection header received
0812 02:42:12.192553 (+ 1798us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:12.192585 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:12.193093 (+ 508us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:12.193553 (+ 460us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:12.194567 (+ 1014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:42:12.194579 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:42:12.194600 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:42:12.194623 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:42:12.194697 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:42:12.194727 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:42:12.194957 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:12.195104 (+ 147us) server_negotiation.cc:299] Negotiation successful
0812 02:42:12.195304 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":630,"thread_start_us":378,"threads_started":1}
I20260812 02:42:12.199941 2181 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e16fd4169098469d914d621f0a221468: Bootstrap starting.
I20260812 02:42:12.200462 7698 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 02:42:12.203219 2181 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e16fd4169098469d914d621f0a221468: Neither blocks nor log segments found. Creating new log.
I20260812 02:42:12.204443 2181 log.cc:824] T 00000000000000000000000000000000 P e16fd4169098469d914d621f0a221468: Log is configured to *not* fsync() on all Append() calls
I20260812 02:42:12.207649 2185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:12.204933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50591 (local address 127.7.132.190:35399)
0812 02:42:12.205124 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:12.205132 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:12.205641 (+ 509us) server_negotiation.cc:407] Connection header received
0812 02:42:12.205818 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:12.205822 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:12.205869 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:12.205948 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:12.206966 (+ 1018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:42:12.206972 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:42:12.206976 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 02:42:12.206998 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:42:12.207048 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 02:42:12.207080 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:42:12.207203 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:12.207311 (+ 108us) server_negotiation.cc:299] Negotiation successful
0812 02:42:12.207369 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 02:42:12.208022 2181 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e16fd4169098469d914d621f0a221468: No bootstrap required, opened a new log
I20260812 02:42:12.215391 2181 raft_consensus.cc:348] T 00000000000000000000000000000000 P e16fd4169098469d914d621f0a221468 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e16fd4169098469d914d621f0a221468" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 35399 } }
I20260812 02:42:12.215797 2181 raft_consensus.cc:374] T 00000000000000000000000000000000 P e16fd4169098469d914d621f0a221468 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 02:42:12.215917 2181 raft_consensus.cc:729] T 00000000000000000000000000000000 P e16fd4169098469d914d621f0a221468 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e16fd4169098469d914d621f0a221468, State: Initialized, Role: FOLLOWER
I20260812 02:42:12.216500 2181 consensus_queue.cc:260] T 00000000000000000000000000000000 P e16fd4169098469d914d621f0a221468 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e16fd4169098469d914d621f0a221468" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 35399 } }
I20260812 02:42:12.216754 2181 raft_consensus.cc:388] T 00000000000000000000000000000000 P e16fd4169098469d914d621f0a221468 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 02:42:12.216938 2181 raft_consensus.cc:482] T 00000000000000000000000000000000 P e16fd4169098469d914d621f0a221468 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 02:42:12.217139 2181 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e16fd4169098469d914d621f0a221468 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:42:12.219450 2181 raft_consensus.cc:504] T 00000000000000000000000000000000 P e16fd4169098469d914d621f0a221468 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e16fd4169098469d914d621f0a221468" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 35399 } }
I20260812 02:42:12.219918 2181 leader_election.cc:302] T 00000000000000000000000000000000 P e16fd4169098469d914d621f0a221468 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e16fd4169098469d914d621f0a221468; no voters:
I20260812 02:42:12.220459 2181 leader_election.cc:288] T 00000000000000000000000000000000 P e16fd4169098469d914d621f0a221468 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 02:42:12.220640 2194 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e16fd4169098469d914d621f0a221468 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:42:12.221462 2194 raft_consensus.cc:686] T 00000000000000000000000000000000 P e16fd4169098469d914d621f0a221468 [term 1 LEADER]: Becoming Leader. State: Replica: e16fd4169098469d914d621f0a221468, State: Running, Role: LEADER
I20260812 02:42:12.222039 2194 consensus_queue.cc:237] T 00000000000000000000000000000000 P e16fd4169098469d914d621f0a221468 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e16fd4169098469d914d621f0a221468" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 35399 } }
I20260812 02:42:12.222921 2181 sys_catalog.cc:564] T 00000000000000000000000000000000 P e16fd4169098469d914d621f0a221468 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:42:12.225183 2196 sys_catalog.cc:455] T 00000000000000000000000000000000 P e16fd4169098469d914d621f0a221468 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e16fd4169098469d914d621f0a221468. Latest consensus state: current_term: 1 leader_uuid: "e16fd4169098469d914d621f0a221468" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e16fd4169098469d914d621f0a221468" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 35399 } } }
I20260812 02:42:12.225720 2196 sys_catalog.cc:458] T 00000000000000000000000000000000 P e16fd4169098469d914d621f0a221468 [sys.catalog]: This master's current role is: LEADER
I20260812 02:42:12.226368 2195 sys_catalog.cc:455] T 00000000000000000000000000000000 P e16fd4169098469d914d621f0a221468 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e16fd4169098469d914d621f0a221468" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e16fd4169098469d914d621f0a221468" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 35399 } } }
I20260812 02:42:12.226729 2195 sys_catalog.cc:458] T 00000000000000000000000000000000 P e16fd4169098469d914d621f0a221468 [sys.catalog]: This master's current role is: LEADER
I20260812 02:42:12.226814 2203 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:42:12.232393 2203 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 02:42:12.242408 2203 catalog_manager.cc:1347] Generated new cluster ID: 0c9f1752653c492da0ab3e5db4789493
I20260812 02:42:12.242558 2203 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 02:42:12.253465 2203 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 02:42:12.253913 2203 catalog_manager.cc:1504] Loading token signing keys...
I20260812 02:42:12.265985 2203 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e16fd4169098469d914d621f0a221468: Generated new TSK 0
I20260812 02:42:12.266978 2203 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 02:42:12.273610 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 2091
W20260812 02:42:12.274854 2171 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:59977
W20260812 02:42:12.275626 7698 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 2091: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 02:42:12.275780 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 2091
2026-08-12T02:42:12Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (2164 ms)
[----------] 18 tests from AuthTokenIssuingTest (39527 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502156971023-7698-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 02:42:12 dist-test-slave-mn4r krb5kdc[2216](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 02:42:12 dist-test-slave-mn4r krb5kdc[2216](info): set up 2 sockets
Aug 12 02:42:12 dist-test-slave-mn4r krb5kdc[2216](info): commencing operation
krb5kdc: starting...
W20260812 02:42:13.583082 7698 mini_kdc.cc:121] Time spent starting KDC: real 1.264s user 0.001s sys 0.047s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502156971023-7698-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502156971023-7698-0/krb5kdc/test-user.keytab.
Aug 12 02:42:13 dist-test-slave-mn4r krb5kdc[2216](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502533, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T02:42:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T02:42:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.132.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
I20260812 02:42:13.925684 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502156971023-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502156971023-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502156971023-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:44621
--webserver_interface=127.7.132.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:40011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.132.190:44621,127.7.132.189:36879,127.7.132.188:33745
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:42:14.290589 2234 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 02:42:14.290935 2234 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:42:14.291002 2234 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:42:14.291093 2234 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:42:14.298453 2234 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:42:14.298552 2234 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:42:14.298609 2234 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:42:14.298660 2234 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:42:14.298722 2234 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:42:14.308583 2234 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:40011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502156971023-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502156971023-7698-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.132.190:44621,127.7.132.189:36879,127.7.132.188:33745
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:44621
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.132.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502156971023-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:42:14.309897 2234 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:42:14.311226 2234 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:42:14.323352 2247 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:42:14.323421 2245 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:42:14.323705 2244 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:42:14.324651 2234 server_base.cc:1034] running on GCE node
I20260812 02:42:14.325446 2234 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:42:14.327008 2234 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:42:14.328301 2234 hybrid_clock.cc:648] HybridClock initialized: now 1786502534328225 us; error 42 us; skew 500 ppm
I20260812 02:42:14.328670 2234 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:42:14 dist-test-slave-mn4r krb5kdc[2216](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502534, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:42:14.332114 2234 init.cc:377] Logged in from keytab as kudu/127.7.132.190@KRBTEST.COM (short username kudu)
I20260812 02:42:14.333791 2234 webserver.cc:460] Webserver started at http://127.7.132.190:39863/ using document root <none> and password file <none>
I20260812 02:42:14.334666 2234 fs_manager.cc:359] Metadata directory not provided
I20260812 02:42:14.334805 2234 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:42:14.335150 2234 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:42:14.338322 2234 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502156971023-7698-0/minicluster-data/master-0/data/instance:
uuid: "104123a083ef4b9eba2f8b669d3d54c4"
format_stamp: "Formatted at 2026-08-12 02:42:14 on dist-test-slave-mn4r"
server_key: "ef6fd4aa6bbf44b010d26100cf7b5c32"
server_key_iv: "cf9df887b643ee8e40181f995d29ba22"
server_key_version: "encryptionkey@0"
I20260812 02:42:14.339151 2234 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502156971023-7698-0/minicluster-data/master-0/wal/instance:
uuid: "104123a083ef4b9eba2f8b669d3d54c4"
format_stamp: "Formatted at 2026-08-12 02:42:14 on dist-test-slave-mn4r"
server_key: "ef6fd4aa6bbf44b010d26100cf7b5c32"
server_key_iv: "cf9df887b643ee8e40181f995d29ba22"
server_key_version: "encryptionkey@0"
I20260812 02:42:14.344843 2234 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260812 02:42:14.348400 2255 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:42:14.350049 2234 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260812 02:42:14.350404 2234 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502156971023-7698-0/minicluster-data/master-0/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502156971023-7698-0/minicluster-data/master-0/wal
uuid: "104123a083ef4b9eba2f8b669d3d54c4"
format_stamp: "Formatted at 2026-08-12 02:42:14 on dist-test-slave-mn4r"
server_key: "ef6fd4aa6bbf44b010d26100cf7b5c32"
server_key_iv: "cf9df887b643ee8e40181f995d29ba22"
server_key_version: "encryptionkey@0"
I20260812 02:42:14.350621 2234 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502156971023-7698-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502156971023-7698-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502156971023-7698-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:42:14.365590 2234 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:42:14.370169 2234 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:42:14.370467 2234 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:42:14.388329 2234 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.190:44621
I20260812 02:42:14.388351 2319 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.190:44621 every 8 connection(s)
I20260812 02:42:14.389756 2234 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
I20260812 02:42:14.393682 2320 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:42:14.394647 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 2234
I20260812 02:42:14.394881 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502156971023-7698-0/minicluster-data/master-0/wal/instance
I20260812 02:42:14.395792 7698 external_mini_cluster.cc:1428] Setting key c545fe8041956e9a3af84b2ae5517618
I20260812 02:42:14.404043 2320 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.132.190" port: 44621 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 02:42:14 dist-test-slave-mn4r krb5kdc[2216](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502534, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.190@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 02:42:14.430852 2322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:14.408273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:44621 (local address 127.0.0.1:44878)
0812 02:42:14.409239 (+ 966us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:14.409295 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0812 02:42:14.411678 (+ 2383us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:42:14.412666 (+ 988us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:42:14.412671 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:42:14.413240 (+ 569us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:42:14.413591 (+ 351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:14.413609 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:14.416190 (+ 2581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:14.416194 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:14.417261 (+ 1067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:14.417274 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:14.418753 (+ 1479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:14.418758 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:14.418884 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:14.419662 (+ 778us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:42:14.419684 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:42:14.421882 (+ 2198us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:42:14.426514 (+ 4632us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:42:14.426524 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:42:14.426535 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:42:14.426950 (+ 415us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:42:14.427455 (+ 505us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:42:14.427458 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:42:14.427461 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:42:14.427631 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:42:14.428770 (+ 1139us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:42:14.428778 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:42:14.428977 (+ 199us) client_negotiation.cc:769] Sending connection context
0812 02:42:14.429207 (+ 230us) client_negotiation.cc:240] Negotiation successful
0812 02:42:14.429661 (+ 454us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":647,"thread_start_us":394,"threads_started":1}
I20260812 02:42:14.430945 2323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:14.408273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44878 (local address 127.7.132.190:44621)
0812 02:42:14.410261 (+ 1988us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:14.410267 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:14.410315 (+ 48us) server_negotiation.cc:407] Connection header received
0812 02:42:14.412084 (+ 1769us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:14.412093 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:14.412408 (+ 315us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:14.412626 (+ 218us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:14.413811 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:14.416020 (+ 2209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:14.417416 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:14.418476 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:14.418682 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:14.422180 (+ 3498us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:42:14.422193 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:42:14.422213 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:42:14.422257 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:42:14.426374 (+ 4117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:14.427108 (+ 734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:14.427121 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:14.427127 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:14.427332 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:14.427747 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:14.427755 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:14.427761 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:14.428574 (+ 813us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:42:14.428795 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:14.429418 (+ 623us) server_negotiation.cc:299] Negotiation successful
0812 02:42:14.429905 (+ 487us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":106,"server-negotiator.queue_time_us":1797,"spinlock_wait_cycles":46336,"thread_start_us":1515,"threads_started":1}
WARNING: no policy specified for kudu/127.7.132.189@KRBTEST.COM; defaulting to no policy
I20260812 02:42:14.434341 2320 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.132.189" port: 36879 } has no permanent_uuid. Determining permanent_uuid...
I20260812 02:42:14.435645 2322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:14.435136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.189:36879 (local address 127.0.0.1:54946)
0812 02:42:14.435331 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:14.435498 (+ 167us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.189:36879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260812 02:42:14.435953 2257 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.132.189:36879: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.189@KRBTEST.COM" created.
I20260812 02:42:14.437860 2322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:14.437339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.189:36879 (local address 127.0.0.1:54958)
0812 02:42:14.437552 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:14.437657 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.189:36879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260812 02:42:14.438169 2320 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.132.189:36879: Network error: Client connection negotiation failed: client connection to 127.7.132.189:36879: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.189.keytab.
Entry for principal kudu/127.7.132.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.189.keytab.
I20260812 02:42:14.475463 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502156971023-7698-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502156971023-7698-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502156971023-7698-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502156971023-7698-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.189:36879
--webserver_interface=127.7.132.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.189.keytab
--principal=kudu/127.7.132.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:40011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.132.190:44621,127.7.132.189:36879,127.7.132.188:33745
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 02:42:14.487561 2320 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.132.189" port: 36879 } attempt: 1
I20260812 02:42:14.488879 2322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:14.488403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.189:36879 (local address 127.0.0.1:54964)
0812 02:42:14.488620 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:14.488736 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.189:36879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260812 02:42:14.490224 2322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:14.489697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.189:36879 (local address 127.0.0.1:54970)
0812 02:42:14.489880 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:14.489978 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.189:36879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260812 02:42:14.490446 2320 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.132.189:36879: Network error: Client connection negotiation failed: client connection to 127.7.132.189:36879: connect: Connection refused (error 111)
I20260812 02:42:14.558725 2320 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.132.189" port: 36879 } attempt: 2
I20260812 02:42:14.560106 2322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:14.559453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.189:36879 (local address 127.0.0.1:54982)
0812 02:42:14.559738 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:14.559876 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.189:36879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260812 02:42:14.561235 2322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:14.560808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.189:36879 (local address 127.0.0.1:54992)
0812 02:42:14.560964 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:14.561080 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.189:36879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260812 02:42:14.561475 2320 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.132.189:36879: Network error: Client connection negotiation failed: client connection to 127.7.132.189:36879: connect: Connection refused (error 111)
I20260812 02:42:14.652791 2320 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.132.189" port: 36879 } attempt: 3
I20260812 02:42:14.654139 2322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:14.653603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.189:36879 (local address 127.0.0.1:55004)
0812 02:42:14.653833 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:14.653933 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.189:36879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260812 02:42:14.655424 2322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:14.655072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.189:36879 (local address 127.0.0.1:55020)
0812 02:42:14.655205 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:14.655280 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.189:36879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260812 02:42:14.655707 2320 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.132.189:36879: Network error: Client connection negotiation failed: client connection to 127.7.132.189:36879: connect: Connection refused (error 111)
I20260812 02:42:14.799018 2320 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.132.189" port: 36879 } attempt: 4
I20260812 02:42:14.800586 2322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:14.799818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.189:36879 (local address 127.0.0.1:55022)
0812 02:42:14.800264 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:14.800395 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.189:36879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":254}
I20260812 02:42:14.802037 2322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:14.801484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.189:36879 (local address 127.0.0.1:55032)
0812 02:42:14.801693 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:14.801818 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.189:36879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260812 02:42:14.802327 2320 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.132.189:36879: Network error: Client connection negotiation failed: client connection to 127.7.132.189:36879: connect: Connection refused (error 111)
W20260812 02:42:14.833316 2327 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 02:42:14.833716 2327 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:42:14.833855 2327 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:42:14.834028 2327 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:42:14.841490 2327 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:42:14.841586 2327 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:42:14.841683 2327 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:42:14.841789 2327 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:42:14.841912 2327 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:42:14.852227 2327 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:40011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502156971023-7698-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502156971023-7698-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.7.132.190:44621,127.7.132.189:36879,127.7.132.188:33745
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.189.keytab
--principal=kudu/127.7.132.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.189:36879
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502156971023-7698-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.7.132.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502156971023-7698-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:42:14.853592 2327 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:42:14.855119 2327 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:42:14.867408 2337 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:42:14.867590 2333 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:42:14.867736 2334 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:42:14.871711 2327 server_base.cc:1034] running on GCE node
I20260812 02:42:14.872579 2327 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:42:14.874028 2327 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:42:14.875308 2327 hybrid_clock.cc:648] HybridClock initialized: now 1786502534875243 us; error 31 us; skew 500 ppm
I20260812 02:42:14.875661 2327 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:42:14 dist-test-slave-mn4r krb5kdc[2216](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502534, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:42:14.878925 2327 init.cc:377] Logged in from keytab as kudu/127.7.132.189@KRBTEST.COM (short username kudu)
I20260812 02:42:14.880528 2327 webserver.cc:460] Webserver started at http://127.7.132.189:38881/ using document root <none> and password file <none>
I20260812 02:42:14.881281 2327 fs_manager.cc:359] Metadata directory not provided
I20260812 02:42:14.881420 2327 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:42:14.881764 2327 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:42:14.884997 2327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502156971023-7698-0/minicluster-data/master-1/data/instance:
uuid: "23f2f837d4d046c5b6688faa53747be7"
format_stamp: "Formatted at 2026-08-12 02:42:14 on dist-test-slave-mn4r"
server_key: "f86e0f4cedcd5fd7683eb23b79489176"
server_key_iv: "33d241fd4fb03d853ed9e4fb1743a3a9"
server_key_version: "encryptionkey@0"
I20260812 02:42:14.885828 2327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502156971023-7698-0/minicluster-data/master-1/wal/instance:
uuid: "23f2f837d4d046c5b6688faa53747be7"
format_stamp: "Formatted at 2026-08-12 02:42:14 on dist-test-slave-mn4r"
server_key: "f86e0f4cedcd5fd7683eb23b79489176"
server_key_iv: "33d241fd4fb03d853ed9e4fb1743a3a9"
server_key_version: "encryptionkey@0"
I20260812 02:42:14.891173 2327 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260812 02:42:14.895251 2345 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:42:14.896749 2327 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.002s
I20260812 02:42:14.896950 2327 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502156971023-7698-0/minicluster-data/master-1/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502156971023-7698-0/minicluster-data/master-1/wal
uuid: "23f2f837d4d046c5b6688faa53747be7"
format_stamp: "Formatted at 2026-08-12 02:42:14 on dist-test-slave-mn4r"
server_key: "f86e0f4cedcd5fd7683eb23b79489176"
server_key_iv: "33d241fd4fb03d853ed9e4fb1743a3a9"
server_key_version: "encryptionkey@0"
I20260812 02:42:14.897179 2327 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502156971023-7698-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502156971023-7698-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502156971023-7698-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:42:14.920364 2327 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:42:14.924642 2327 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:42:14.924935 2327 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:42:14.944911 2327 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.189:36879
I20260812 02:42:14.944931 2412 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.189:36879 every 8 connection(s)
I20260812 02:42:14.946480 2327 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502156971023-7698-0/minicluster-data/master-1/data/info.pb
I20260812 02:42:14.950250 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 2327
I20260812 02:42:14.950466 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502156971023-7698-0/minicluster-data/master-1/wal/instance
I20260812 02:42:14.950591 2413 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:42:14.951074 7698 external_mini_cluster.cc:1428] Setting key d2442566c7e775fd421498115362bb5c
I20260812 02:42:14.960803 2413 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.132.190" port: 44621 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 02:42:14 dist-test-slave-mn4r krb5kdc[2216](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502534, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.189@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 02:42:14.985342 2415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:14.964330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44884 (local address 127.7.132.190:44621)
0812 02:42:14.964892 (+ 562us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:14.964899 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:14.966003 (+ 1104us) server_negotiation.cc:407] Connection header received
0812 02:42:14.968130 (+ 2127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:14.968135 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:14.968205 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:14.968296 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:14.970495 (+ 2199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:14.971232 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:14.973769 (+ 2537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:14.974253 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:14.974352 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:14.978866 (+ 4514us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:42:14.978871 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:42:14.978875 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:42:14.978923 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:42:14.981800 (+ 2877us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:14.982867 (+ 1067us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:14.982877 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:14.982880 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:14.983010 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:14.983480 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:14.983483 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:14.983485 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:14.983779 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:42:14.983944 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:14.984710 (+ 766us) server_negotiation.cc:299] Negotiation successful
0812 02:42:14.984948 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":407,"thread_start_us":257,"threads_started":1}
WARNING: no policy specified for kudu/127.7.132.188@KRBTEST.COM; defaulting to no policy
I20260812 02:42:14.986507 2416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:14.964943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:44621 (local address 127.0.0.1:44884)
0812 02:42:14.965808 (+ 865us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:14.965853 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 02:42:14.967808 (+ 1955us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:42:14.968574 (+ 766us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:42:14.968588 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:42:14.969398 (+ 810us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:42:14.970244 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:14.970275 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:14.971410 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:14.971418 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:14.973617 (+ 2199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:14.973636 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:14.974493 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:14.974504 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:14.974876 (+ 372us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:14.975674 (+ 798us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:42:14.975715 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:42:14.978597 (+ 2882us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:42:14.982032 (+ 3435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:42:14.982049 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:42:14.982066 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:42:14.982721 (+ 655us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:42:14.983133 (+ 412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:42:14.983138 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:42:14.983142 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:42:14.983340 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:42:14.983917 (+ 577us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:42:14.983925 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:42:14.984331 (+ 406us) client_negotiation.cc:769] Sending connection context
0812 02:42:14.984614 (+ 283us) client_negotiation.cc:240] Negotiation successful
0812 02:42:14.985192 (+ 578us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":591,"thread_start_us":375,"threads_started":1}
I20260812 02:42:14.988348 2413 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.132.189" port: 36879 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.188@KRBTEST.COM" created.
Aug 12 02:42:14 dist-test-slave-mn4r krb5kdc[2216](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502534, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.189@KRBTEST.COM for kudu/127.7.132.189@KRBTEST.COM
I20260812 02:42:15.003575 2416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:14.989197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.189:36879 (local address 127.0.0.1:55046)
0812 02:42:14.989412 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:14.989448 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 02:42:14.989572 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:42:14.990663 (+ 1091us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:42:14.990667 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:42:14.991119 (+ 452us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:42:14.991252 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:14.991261 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:14.992686 (+ 1425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:14.992693 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:14.993537 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:14.993548 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:14.994467 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:14.994473 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:14.994642 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:14.995251 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:42:14.995268 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:42:14.997520 (+ 2252us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:42:15.000921 (+ 3401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:42:15.000926 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:42:15.000933 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:42:15.001378 (+ 445us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:42:15.001768 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:42:15.001772 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:42:15.001775 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:42:15.001930 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:42:15.002595 (+ 665us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:42:15.002603 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:42:15.002745 (+ 142us) client_negotiation.cc:769] Sending connection context
0812 02:42:15.002900 (+ 155us) client_negotiation.cc:240] Negotiation successful
0812 02:42:15.003177 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260812 02:42:15.003674 2420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:14.989456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55046 (local address 127.7.132.189:36879)
0812 02:42:14.990236 (+ 780us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:14.990247 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:14.990274 (+ 27us) server_negotiation.cc:407] Connection header received
0812 02:42:14.990317 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:14.990323 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:14.990507 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:14.990671 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:14.991428 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:14.992552 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:14.993683 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:14.994297 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:14.994438 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:14.997725 (+ 3287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:42:14.997737 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:42:14.997750 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:42:14.997788 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:42:15.000789 (+ 3001us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:15.001527 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:15.001541 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:15.001543 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:15.001652 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:15.002046 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:15.002052 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:15.002055 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:15.002455 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:42:15.002674 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:15.003056 (+ 382us) server_negotiation.cc:299] Negotiation successful
0812 02:42:15.003288 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":569,"thread_start_us":363,"threads_started":1}
I20260812 02:42:15.006417 2413 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.132.188" port: 33745 } has no permanent_uuid. Determining permanent_uuid...
I20260812 02:42:15.007727 2416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:15.007217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:33745 (local address 127.0.0.1:42022)
0812 02:42:15.007416 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:15.007546 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.188:33745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
W20260812 02:42:15.008044 2346 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.132.188:33745: connect: Connection refused (error 111)
I20260812 02:42:15.009754 2416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:15.009390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:33745 (local address 127.0.0.1:42038)
0812 02:42:15.009540 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:15.009640 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.188:33745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260812 02:42:15.010000 2413 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.132.188:33745: Network error: Client connection negotiation failed: client connection to 127.7.132.188:33745: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.188.keytab.
Entry for principal kudu/127.7.132.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.188.keytab.
I20260812 02:42:15.029731 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502156971023-7698-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502156971023-7698-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502156971023-7698-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502156971023-7698-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.188:33745
--webserver_interface=127.7.132.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.188.keytab
--principal=kudu/127.7.132.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:40011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.132.190:44621,127.7.132.189:36879,127.7.132.188:33745
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 02:42:15.059401 2413 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.132.188" port: 33745 } attempt: 1
I20260812 02:42:15.060711 2416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:15.060227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:33745 (local address 127.0.0.1:42050)
0812 02:42:15.060434 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:15.060545 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.188:33745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260812 02:42:15.062053 2416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:15.061654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:33745 (local address 127.0.0.1:42066)
0812 02:42:15.061845 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:15.061940 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.188:33745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260812 02:42:15.062422 2413 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.132.188:33745: Network error: Client connection negotiation failed: client connection to 127.7.132.188:33745: connect: Connection refused (error 111)
I20260812 02:42:15.101678 2320 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.132.189" port: 36879 } attempt: 5
Aug 12 02:42:15 dist-test-slave-mn4r krb5kdc[2216](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502534, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.190@KRBTEST.COM for kudu/127.7.132.189@KRBTEST.COM
I20260812 02:42:15.117161 2322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:15.102524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.189:36879 (local address 127.0.0.1:55054)
0812 02:42:15.102731 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:15.102755 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0812 02:42:15.102877 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:42:15.103285 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:42:15.103289 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:42:15.103684 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:42:15.103872 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:15.103881 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:15.105327 (+ 1446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:15.105331 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:15.106357 (+ 1026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:15.106371 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:15.107377 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:15.107381 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:15.107563 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:15.108257 (+ 694us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:42:15.108276 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:42:15.110658 (+ 2382us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:42:15.114780 (+ 4122us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:42:15.114784 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:42:15.114787 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:42:15.115201 (+ 414us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:42:15.115603 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:42:15.115606 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:42:15.115609 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:42:15.115737 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:42:15.116386 (+ 649us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:42:15.116390 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:42:15.116530 (+ 140us) client_negotiation.cc:769] Sending connection context
0812 02:42:15.116651 (+ 121us) client_negotiation.cc:240] Negotiation successful
0812 02:42:15.116900 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":69}
I20260812 02:42:15.117331 2420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:15.102617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55054 (local address 127.7.132.189:36879)
0812 02:42:15.102844 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:15.102850 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:15.102878 (+ 28us) server_negotiation.cc:407] Connection header received
0812 02:42:15.103017 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:15.103023 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:15.103113 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:15.103244 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:15.104037 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:15.105152 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:15.106523 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:15.107211 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:15.107348 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:15.110900 (+ 3552us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:42:15.110908 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:42:15.110912 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:42:15.110955 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:42:15.114637 (+ 3682us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:15.115327 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:15.115333 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:15.115336 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:15.115489 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:15.115845 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:15.115852 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:15.115855 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:15.116259 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:42:15.116407 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:15.116795 (+ 388us) server_negotiation.cc:299] Negotiation successful
0812 02:42:15.117031 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260812 02:42:15.118417 2320 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.132.188" port: 33745 } has no permanent_uuid. Determining permanent_uuid...
I20260812 02:42:15.119567 2322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:15.119165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:33745 (local address 127.0.0.1:42076)
0812 02:42:15.119339 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:15.119428 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.188:33745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260812 02:42:15.120529 2322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:15.120223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:33745 (local address 127.0.0.1:42082)
0812 02:42:15.120357 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:15.120435 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.188:33745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260812 02:42:15.120817 2320 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.132.188:33745: Network error: Client connection negotiation failed: client connection to 127.7.132.188:33745: connect: Connection refused (error 111)
I20260812 02:42:15.130656 2413 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.132.188" port: 33745 } attempt: 2
I20260812 02:42:15.131839 2416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:15.131380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:33745 (local address 127.0.0.1:42088)
0812 02:42:15.131590 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:15.131685 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.188:33745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260812 02:42:15.132833 2416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:15.132486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:33745 (local address 127.0.0.1:42104)
0812 02:42:15.132651 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:15.132719 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.188:33745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260812 02:42:15.133051 2413 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.132.188:33745: Network error: Client connection negotiation failed: client connection to 127.7.132.188:33745: connect: Connection refused (error 111)
I20260812 02:42:15.172071 2320 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.132.188" port: 33745 } attempt: 1
I20260812 02:42:15.173451 2322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:15.172943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:33745 (local address 127.0.0.1:42118)
0812 02:42:15.173178 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:15.173275 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.188:33745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260812 02:42:15.174588 2322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:15.174243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:33745 (local address 127.0.0.1:42124)
0812 02:42:15.174396 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:15.174481 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.188:33745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260812 02:42:15.174805 2320 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.132.188:33745: Network error: Client connection negotiation failed: client connection to 127.7.132.188:33745: connect: Connection refused (error 111)
I20260812 02:42:15.224264 2413 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.132.188" port: 33745 } attempt: 3
I20260812 02:42:15.225497 2416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:15.225021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:33745 (local address 127.0.0.1:42136)
0812 02:42:15.225246 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:15.225341 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.188:33745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260812 02:42:15.226706 2416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:15.226263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:33745 (local address 127.0.0.1:42152)
0812 02:42:15.226501 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:15.226587 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.188:33745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132}
W20260812 02:42:15.226908 2413 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.132.188:33745: Network error: Client connection negotiation failed: client connection to 127.7.132.188:33745: connect: Connection refused (error 111)
I20260812 02:42:15.243005 2320 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.132.188" port: 33745 } attempt: 2
I20260812 02:42:15.244230 2322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:15.243700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:33745 (local address 127.0.0.1:42162)
0812 02:42:15.243929 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:15.244038 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.188:33745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260812 02:42:15.245261 2322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:15.244906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:33745 (local address 127.0.0.1:42174)
0812 02:42:15.245076 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:15.245151 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.188:33745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260812 02:42:15.245476 2320 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.132.188:33745: Network error: Client connection negotiation failed: client connection to 127.7.132.188:33745: connect: Connection refused (error 111)
I20260812 02:42:15.351672 2320 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.132.188" port: 33745 } attempt: 3
I20260812 02:42:15.352861 2322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:15.352382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:33745 (local address 127.0.0.1:42188)
0812 02:42:15.352589 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:15.352692 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.188:33745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260812 02:42:15.353983 2322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:15.353547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:33745 (local address 127.0.0.1:42190)
0812 02:42:15.353773 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:15.353876 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.188:33745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101}
W20260812 02:42:15.354193 2320 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.132.188:33745: Network error: Client connection negotiation failed: client connection to 127.7.132.188:33745: connect: Connection refused (error 111)
I20260812 02:42:15.370163 2413 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.132.188" port: 33745 } attempt: 4
I20260812 02:42:15.371358 2416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:15.370894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:33745 (local address 127.0.0.1:42192)
0812 02:42:15.371121 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:15.371209 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.188:33745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260812 02:42:15.372370 2416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:15.372024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:33745 (local address 127.0.0.1:42196)
0812 02:42:15.372180 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:15.372262 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.188:33745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260812 02:42:15.372562 2413 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.132.188:33745: Network error: Client connection negotiation failed: client connection to 127.7.132.188:33745: connect: Connection refused (error 111)
W20260812 02:42:15.394680 2424 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 02:42:15.395058 2424 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:42:15.395183 2424 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:42:15.395319 2424 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:42:15.402789 2424 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:42:15.402906 2424 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:42:15.403033 2424 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:42:15.403121 2424 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:42:15.403229 2424 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:42:15.413177 2424 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:40011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502156971023-7698-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502156971023-7698-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.7.132.190:44621,127.7.132.189:36879,127.7.132.188:33745
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.188.keytab
--principal=kudu/127.7.132.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.188:33745
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502156971023-7698-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.7.132.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502156971023-7698-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:42:15.414562 2424 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:42:15.415944 2424 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:42:15.428300 2430 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:42:15.428262 2431 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:42:15.428213 2434 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:42:15.429368 2424 server_base.cc:1034] running on GCE node
I20260812 02:42:15.430212 2424 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:42:15.431654 2424 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:42:15.432890 2424 hybrid_clock.cc:648] HybridClock initialized: now 1786502535432801 us; error 58 us; skew 500 ppm
I20260812 02:42:15.433262 2424 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:42:15 dist-test-slave-mn4r krb5kdc[2216](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502535, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:42:15.436674 2424 init.cc:377] Logged in from keytab as kudu/127.7.132.188@KRBTEST.COM (short username kudu)
I20260812 02:42:15.438350 2424 webserver.cc:460] Webserver started at http://127.7.132.188:42881/ using document root <none> and password file <none>
I20260812 02:42:15.439106 2424 fs_manager.cc:359] Metadata directory not provided
I20260812 02:42:15.439244 2424 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:42:15.439592 2424 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:42:15.442839 2424 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502156971023-7698-0/minicluster-data/master-2/data/instance:
uuid: "d537441aec13451f9c53abb19ee7f48e"
format_stamp: "Formatted at 2026-08-12 02:42:15 on dist-test-slave-mn4r"
server_key: "c32e6d3158cd9fbb19ac1189d0f05896"
server_key_iv: "4ceeabbf409fc8f37f77a60e82a60b9d"
server_key_version: "encryptionkey@0"
I20260812 02:42:15.443701 2424 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502156971023-7698-0/minicluster-data/master-2/wal/instance:
uuid: "d537441aec13451f9c53abb19ee7f48e"
format_stamp: "Formatted at 2026-08-12 02:42:15 on dist-test-slave-mn4r"
server_key: "c32e6d3158cd9fbb19ac1189d0f05896"
server_key_iv: "4ceeabbf409fc8f37f77a60e82a60b9d"
server_key_version: "encryptionkey@0"
I20260812 02:42:15.449419 2424 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260812 02:42:15.453293 2441 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:42:15.454906 2424 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 02:42:15.455087 2424 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502156971023-7698-0/minicluster-data/master-2/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502156971023-7698-0/minicluster-data/master-2/wal
uuid: "d537441aec13451f9c53abb19ee7f48e"
format_stamp: "Formatted at 2026-08-12 02:42:15 on dist-test-slave-mn4r"
server_key: "c32e6d3158cd9fbb19ac1189d0f05896"
server_key_iv: "4ceeabbf409fc8f37f77a60e82a60b9d"
server_key_version: "encryptionkey@0"
I20260812 02:42:15.455318 2424 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502156971023-7698-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502156971023-7698-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502156971023-7698-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:42:15.468091 2424 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:42:15.472354 2424 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:42:15.472657 2424 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:42:15.491827 2424 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.188:33745
I20260812 02:42:15.491845 2503 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.188:33745 every 8 connection(s)
I20260812 02:42:15.493295 2424 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502156971023-7698-0/minicluster-data/master-2/data/info.pb
I20260812 02:42:15.497262 2504 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:42:15.503181 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 2424
I20260812 02:42:15.503392 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786502156971023-7698-0/minicluster-data/master-2/wal/instance
I20260812 02:42:15.503854 7698 external_mini_cluster.cc:1428] Setting key e904471b72e7b59133863ba3fada72bc
I20260812 02:42:15.504885 2504 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.132.190" port: 44621 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 02:42:15 dist-test-slave-mn4r krb5kdc[2216](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502535, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.188@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
Aug 12 02:42:15 dist-test-slave-mn4r krb5kdc[2216](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502533, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.132.188@KRBTEST.COM
I20260812 02:42:15.526834 2509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:15.508166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44886 (local address 127.7.132.190:44621)
0812 02:42:15.508666 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:15.508672 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:15.509282 (+ 610us) server_negotiation.cc:407] Connection header received
0812 02:42:15.509729 (+ 447us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:15.509738 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:15.509833 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:15.509953 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:15.511801 (+ 1848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:15.512945 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:15.515125 (+ 2180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:15.515780 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:15.515913 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:15.520464 (+ 4551us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:42:15.520470 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:42:15.520473 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:42:15.520515 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:42:15.523537 (+ 3022us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:15.524273 (+ 736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:15.524278 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:15.524288 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:15.524415 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:15.524862 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:15.524866 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:15.524868 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:15.525149 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:42:15.525308 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:15.525941 (+ 633us) server_negotiation.cc:299] Negotiation successful
0812 02:42:15.526211 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":377,"thread_start_us":247,"threads_started":1}
I20260812 02:42:15.527796 2511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:15.508160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:44621 (local address 127.0.0.1:44886)
0812 02:42:15.509128 (+ 968us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:15.509171 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 02:42:15.509480 (+ 309us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:42:15.510108 (+ 628us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:42:15.510119 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:42:15.511039 (+ 920us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:42:15.511611 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:15.511645 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:15.513130 (+ 1485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:15.513133 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:15.514958 (+ 1825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:15.514975 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:15.516023 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:15.516028 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:15.516218 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:15.517092 (+ 874us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:42:15.517124 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:42:15.520256 (+ 3132us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:42:15.523683 (+ 3427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:42:15.523695 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:42:15.523707 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:42:15.524140 (+ 433us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:42:15.524541 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:42:15.524546 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:42:15.524549 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:42:15.524728 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:42:15.525285 (+ 557us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:42:15.525293 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:42:15.525576 (+ 283us) client_negotiation.cc:769] Sending connection context
0812 02:42:15.525838 (+ 262us) client_negotiation.cc:240] Negotiation successful
0812 02:42:15.526440 (+ 602us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":707,"thread_start_us":537,"threads_started":1}
I20260812 02:42:15.527817 2508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:15.505900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42210 (local address 127.7.132.188:33745)
0812 02:42:15.506803 (+ 903us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:15.506819 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:15.506871 (+ 52us) server_negotiation.cc:407] Connection header received
0812 02:42:15.509166 (+ 2295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:15.509190 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:15.509583 (+ 393us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:15.509812 (+ 229us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:15.511781 (+ 1969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:15.513414 (+ 1633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:15.515256 (+ 1842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:15.516128 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:15.516223 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:15.521012 (+ 4789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:42:15.521021 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:42:15.521034 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:42:15.521070 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:42:15.523793 (+ 2723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:15.524476 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:15.524483 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:15.524485 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:15.524641 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:15.525039 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:15.525043 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:15.525045 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:15.525575 (+ 530us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:42:15.525909 (+ 334us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:15.526256 (+ 347us) server_negotiation.cc:299] Negotiation successful
0812 02:42:15.526575 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":38,"server-negotiator.queue_time_us":669,"spinlock_wait_cycles":26880,"thread_start_us":327,"threads_started":1}
I20260812 02:42:15.529467 2504 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.132.189" port: 36879 } has no permanent_uuid. Determining permanent_uuid...
I20260812 02:42:15.531412 2320 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.132.188" port: 33745 } attempt: 4
Aug 12 02:42:15 dist-test-slave-mn4r krb5kdc[2216](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502535, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.188@KRBTEST.COM for kudu/127.7.132.189@KRBTEST.COM
Aug 12 02:42:15 dist-test-slave-mn4r krb5kdc[2216](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502533, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
Aug 12 02:42:15 dist-test-slave-mn4r krb5kdc[2216](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502534, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.190@KRBTEST.COM for kudu/127.7.132.188@KRBTEST.COM
I20260812 02:42:15.545522 2511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:15.530431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.189:36879 (local address 127.0.0.1:55066)
0812 02:42:15.530579 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:15.530599 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 02:42:15.530707 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:42:15.531062 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:42:15.531066 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:42:15.531574 (+ 508us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:42:15.531776 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:15.531788 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:15.532996 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:15.533003 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:15.534535 (+ 1532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:15.534550 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:15.535492 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:15.535497 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:15.535707 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:15.536311 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:42:15.536328 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:42:15.538848 (+ 2520us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:42:15.542691 (+ 3843us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:42:15.542697 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:42:15.542701 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:42:15.543321 (+ 620us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:42:15.543741 (+ 420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:42:15.543746 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:42:15.543750 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:42:15.543930 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:42:15.544475 (+ 545us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:42:15.544480 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:42:15.544644 (+ 164us) client_negotiation.cc:769] Sending connection context
0812 02:42:15.544789 (+ 145us) client_negotiation.cc:240] Negotiation successful
0812 02:42:15.545141 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260812 02:42:15.545539 2420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:15.530359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55066 (local address 127.7.132.189:36879)
0812 02:42:15.530594 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:15.530601 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:15.530664 (+ 63us) server_negotiation.cc:407] Connection header received
0812 02:42:15.530826 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:15.530831 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:15.530921 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:15.531001 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:15.531940 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:15.532848 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:15.534686 (+ 1838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:15.535343 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:15.535484 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:15.539075 (+ 3591us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:42:15.539083 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:42:15.539089 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:42:15.539131 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:42:15.542551 (+ 3420us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:15.543458 (+ 907us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:15.543464 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:15.543468 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:15.543622 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:15.544054 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:15.544059 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:15.544063 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:15.544351 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:42:15.544511 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:15.544891 (+ 380us) server_negotiation.cc:299] Negotiation successful
0812 02:42:15.545122 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260812 02:42:15.547204 2504 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.132.188" port: 33745 } has no permanent_uuid. Determining permanent_uuid...
I20260812 02:42:15.549822 2509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:15.531574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44900 (local address 127.7.132.190:44621)
0812 02:42:15.531751 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:15.531756 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:15.531774 (+ 18us) server_negotiation.cc:407] Connection header received
0812 02:42:15.531834 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:15.531841 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:15.531946 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:15.532068 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:15.533528 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:15.534653 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:15.536229 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:15.536994 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:15.537105 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:15.542206 (+ 5101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:42:15.542212 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:42:15.542217 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:42:15.542255 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:42:15.546189 (+ 3934us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:15.547015 (+ 826us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:15.547022 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:15.547028 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:15.547169 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:15.547602 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:15.547607 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:15.547613 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:15.547983 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:42:15.548294 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:15.549196 (+ 902us) server_negotiation.cc:299] Negotiation successful
0812 02:42:15.549441 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":28,"server-negotiator.queue_time_us":63}
I20260812 02:42:15.550770 2322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:15.531996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:33745 (local address 127.0.0.1:42220)
0812 02:42:15.533051 (+ 1055us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:15.533074 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0812 02:42:15.533190 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:42:15.533502 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:42:15.533506 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:42:15.533889 (+ 383us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:42:15.534049 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:15.534058 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:15.535351 (+ 1293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:15.535357 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:15.536895 (+ 1538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:15.536911 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:15.538169 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:15.538177 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:15.538335 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:15.538882 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:42:15.538899 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:42:15.541941 (+ 3042us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:42:15.546922 (+ 4981us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:42:15.546929 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:42:15.546936 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:42:15.547492 (+ 556us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:42:15.548675 (+ 1183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:42:15.548679 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:42:15.548683 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:42:15.548850 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:42:15.549723 (+ 873us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:42:15.549728 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:42:15.549957 (+ 229us) client_negotiation.cc:769] Sending connection context
0812 02:42:15.550144 (+ 187us) client_negotiation.cc:240] Negotiation successful
0812 02:42:15.550360 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":963,"mutex_wait_us":19}
I20260812 02:42:15.550843 2508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:15.532399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42220 (local address 127.7.132.188:33745)
0812 02:42:15.532574 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:15.532579 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:15.533144 (+ 565us) server_negotiation.cc:407] Connection header received
0812 02:42:15.533301 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:15.533305 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:15.533369 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:15.533482 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:15.534236 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:15.535150 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:15.537075 (+ 1925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:15.537646 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:15.538056 (+ 410us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:15.542211 (+ 4155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:42:15.542217 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:42:15.542221 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:42:15.542254 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:42:15.546780 (+ 4526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:15.548369 (+ 1589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:15.548374 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:15.548378 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:15.548538 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:15.549172 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:15.549178 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:15.549181 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:15.549569 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:42:15.549722 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:15.550223 (+ 501us) server_negotiation.cc:299] Negotiation successful
0812 02:42:15.550464 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 02:42:15.557782 2320 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 104123a083ef4b9eba2f8b669d3d54c4: Bootstrap starting.
Aug 12 02:42:15 dist-test-slave-mn4r krb5kdc[2216](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502535, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.188@KRBTEST.COM for kudu/127.7.132.188@KRBTEST.COM
Aug 12 02:42:15 dist-test-slave-mn4r krb5kdc[2216](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502533, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.132.189@KRBTEST.COM
I20260812 02:42:15.562212 2320 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 104123a083ef4b9eba2f8b669d3d54c4: Neither blocks nor log segments found. Creating new log.
I20260812 02:42:15.563575 2320 log.cc:824] T 00000000000000000000000000000000 P 104123a083ef4b9eba2f8b669d3d54c4: Log is configured to *not* fsync() on all Append() calls
I20260812 02:42:15.565485 2511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:15.547968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:33745 (local address 127.0.0.1:42228)
0812 02:42:15.548612 (+ 644us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:15.548631 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 02:42:15.548738 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:42:15.549025 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:42:15.549029 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:42:15.549523 (+ 494us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:42:15.549690 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:15.549702 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:15.552883 (+ 3181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:15.552888 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:15.554303 (+ 1415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:15.554317 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:15.555382 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:15.555389 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:15.555612 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:15.556253 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:42:15.556273 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:42:15.558927 (+ 2654us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:42:15.562772 (+ 3845us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:42:15.562779 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:42:15.562782 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:42:15.563310 (+ 528us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:42:15.563771 (+ 461us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:42:15.563779 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:42:15.563783 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:42:15.563953 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:42:15.564642 (+ 689us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:42:15.564649 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:42:15.564810 (+ 161us) client_negotiation.cc:769] Sending connection context
0812 02:42:15.564959 (+ 149us) client_negotiation.cc:240] Negotiation successful
0812 02:42:15.565237 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":524}
I20260812 02:42:15.566476 2512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:15.548016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42228 (local address 127.7.132.188:33745)
0812 02:42:15.548740 (+ 724us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:15.548745 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:15.548767 (+ 22us) server_negotiation.cc:407] Connection header received
0812 02:42:15.548841 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:15.548846 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:15.548901 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:15.549007 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:15.549837 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:15.552745 (+ 2908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:15.554461 (+ 1716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:15.555073 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:15.555208 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:15.559166 (+ 3958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:42:15.559175 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:42:15.559182 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:42:15.559221 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:42:15.562643 (+ 3422us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:15.563447 (+ 804us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:15.563453 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:15.563456 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:15.563629 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:15.564079 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:15.564085 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:15.564088 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:15.564499 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:42:15.565877 (+ 1378us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:15.565988 (+ 111us) server_negotiation.cc:299] Negotiation successful
0812 02:42:15.566216 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":618,"thread_start_us":337,"threads_started":1}
I20260812 02:42:15.567265 2320 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 104123a083ef4b9eba2f8b669d3d54c4: No bootstrap required, opened a new log
I20260812 02:42:15.570267 2420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:15.552305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55070 (local address 127.7.132.189:36879)
0812 02:42:15.552464 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:15.552469 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:15.552586 (+ 117us) server_negotiation.cc:407] Connection header received
0812 02:42:15.552773 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:15.552781 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:15.552867 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:15.552959 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:15.554398 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:15.555295 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:15.556751 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:15.557184 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:15.557402 (+ 218us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:15.562971 (+ 5569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:42:15.562978 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:42:15.562982 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:42:15.563024 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:42:15.566672 (+ 3648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:15.567517 (+ 845us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:15.567523 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:15.567526 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:15.567721 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:15.568280 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:15.568300 (+ 20us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:15.568304 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:15.568762 (+ 458us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:42:15.569056 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:15.569634 (+ 578us) server_negotiation.cc:299] Negotiation successful
0812 02:42:15.569889 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 02:42:15.572203 2504 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d537441aec13451f9c53abb19ee7f48e: Bootstrap starting.
I20260812 02:42:15.576164 2504 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d537441aec13451f9c53abb19ee7f48e: Neither blocks nor log segments found. Creating new log.
I20260812 02:42:15.576248 7698 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 02:42:15.576766 2320 raft_consensus.cc:348] T 00000000000000000000000000000000 P 104123a083ef4b9eba2f8b669d3d54c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "104123a083ef4b9eba2f8b669d3d54c4" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 44621 } } peers { permanent_uuid: "23f2f837d4d046c5b6688faa53747be7" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 36879 } } peers { permanent_uuid: "d537441aec13451f9c53abb19ee7f48e" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 33745 } }
I20260812 02:42:15.577159 2320 raft_consensus.cc:729] T 00000000000000000000000000000000 P 104123a083ef4b9eba2f8b669d3d54c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 104123a083ef4b9eba2f8b669d3d54c4, State: Initialized, Role: FOLLOWER
I20260812 02:42:15.577361 2504 log.cc:824] T 00000000000000000000000000000000 P d537441aec13451f9c53abb19ee7f48e: Log is configured to *not* fsync() on all Append() calls
I20260812 02:42:15.577654 2320 consensus_queue.cc:260] T 00000000000000000000000000000000 P 104123a083ef4b9eba2f8b669d3d54c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "104123a083ef4b9eba2f8b669d3d54c4" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 44621 } } peers { permanent_uuid: "23f2f837d4d046c5b6688faa53747be7" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 36879 } } peers { permanent_uuid: "d537441aec13451f9c53abb19ee7f48e" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 33745 } }
I20260812 02:42:15.580015 2516 sys_catalog.cc:455] T 00000000000000000000000000000000 P 104123a083ef4b9eba2f8b669d3d54c4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "104123a083ef4b9eba2f8b669d3d54c4" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 44621 } } peers { permanent_uuid: "23f2f837d4d046c5b6688faa53747be7" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 36879 } } peers { permanent_uuid: "d537441aec13451f9c53abb19ee7f48e" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 33745 } } }
I20260812 02:42:15.580195 2288 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "104123a083ef4b9eba2f8b669d3d54c4"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44900
I20260812 02:42:15.580549 2516 sys_catalog.cc:458] T 00000000000000000000000000000000 P 104123a083ef4b9eba2f8b669d3d54c4 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 02:42:15.580708 2504 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d537441aec13451f9c53abb19ee7f48e: No bootstrap required, opened a new log
I20260812 02:42:15.581220 2320 sys_catalog.cc:564] T 00000000000000000000000000000000 P 104123a083ef4b9eba2f8b669d3d54c4 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:42:15.581388 2288 raft_consensus.cc:482] T 00000000000000000000000000000000 P 104123a083ef4b9eba2f8b669d3d54c4 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 02:42:15.581606 2288 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 104123a083ef4b9eba2f8b669d3d54c4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:42:15.585482 2288 raft_consensus.cc:504] T 00000000000000000000000000000000 P 104123a083ef4b9eba2f8b669d3d54c4 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "104123a083ef4b9eba2f8b669d3d54c4" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 44621 } } peers { permanent_uuid: "23f2f837d4d046c5b6688faa53747be7" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 36879 } } peers { permanent_uuid: "d537441aec13451f9c53abb19ee7f48e" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 33745 } }
I20260812 02:42:15.589057 2288 leader_election.cc:288] T 00000000000000000000000000000000 P 104123a083ef4b9eba2f8b669d3d54c4 [CANDIDATE]: Term 1 election: Requested vote from peers 23f2f837d4d046c5b6688faa53747be7 (127.7.132.189:36879), d537441aec13451f9c53abb19ee7f48e (127.7.132.188:33745)
I20260812 02:42:15.591876 2504 raft_consensus.cc:348] T 00000000000000000000000000000000 P d537441aec13451f9c53abb19ee7f48e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "104123a083ef4b9eba2f8b669d3d54c4" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 44621 } } peers { permanent_uuid: "23f2f837d4d046c5b6688faa53747be7" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 36879 } } peers { permanent_uuid: "d537441aec13451f9c53abb19ee7f48e" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 33745 } }
I20260812 02:42:15.592437 2504 raft_consensus.cc:729] T 00000000000000000000000000000000 P d537441aec13451f9c53abb19ee7f48e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d537441aec13451f9c53abb19ee7f48e, State: Initialized, Role: FOLLOWER
I20260812 02:42:15.593075 2504 consensus_queue.cc:260] T 00000000000000000000000000000000 P d537441aec13451f9c53abb19ee7f48e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "104123a083ef4b9eba2f8b669d3d54c4" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 44621 } } peers { permanent_uuid: "23f2f837d4d046c5b6688faa53747be7" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 36879 } } peers { permanent_uuid: "d537441aec13451f9c53abb19ee7f48e" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 33745 } }
I20260812 02:42:15.600188 2537 sys_catalog.cc:455] T 00000000000000000000000000000000 P d537441aec13451f9c53abb19ee7f48e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "104123a083ef4b9eba2f8b669d3d54c4" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 44621 } } peers { permanent_uuid: "23f2f837d4d046c5b6688faa53747be7" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 36879 } } peers { permanent_uuid: "d537441aec13451f9c53abb19ee7f48e" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 33745 } } }
I20260812 02:42:15.600677 2537 sys_catalog.cc:458] T 00000000000000000000000000000000 P d537441aec13451f9c53abb19ee7f48e [sys.catalog]: This master's current role is: FOLLOWER
I20260812 02:42:15.602984 2504 sys_catalog.cc:564] T 00000000000000000000000000000000 P d537441aec13451f9c53abb19ee7f48e [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:42:15.603052 2473 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "104123a083ef4b9eba2f8b669d3d54c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "d537441aec13451f9c53abb19ee7f48e"
I20260812 02:42:15.603541 2473 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d537441aec13451f9c53abb19ee7f48e [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:42:15.608191 2473 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d537441aec13451f9c53abb19ee7f48e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 104123a083ef4b9eba2f8b669d3d54c4 in term 1.
W20260812 02:42:15.609082 2533 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 104123a083ef4b9eba2f8b669d3d54c4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 02:42:15.609362 2533 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 02:42:15.609200 2256 leader_election.cc:302] T 00000000000000000000000000000000 P 104123a083ef4b9eba2f8b669d3d54c4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 104123a083ef4b9eba2f8b669d3d54c4, d537441aec13451f9c53abb19ee7f48e; no voters:
I20260812 02:42:15.609974 2516 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 104123a083ef4b9eba2f8b669d3d54c4 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:42:15.611498 2516 raft_consensus.cc:686] T 00000000000000000000000000000000 P 104123a083ef4b9eba2f8b669d3d54c4 [term 1 LEADER]: Becoming Leader. State: Replica: 104123a083ef4b9eba2f8b669d3d54c4, State: Running, Role: LEADER
I20260812 02:42:15.612222 2516 consensus_queue.cc:237] T 00000000000000000000000000000000 P 104123a083ef4b9eba2f8b669d3d54c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "104123a083ef4b9eba2f8b669d3d54c4" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 44621 } } peers { permanent_uuid: "23f2f837d4d046c5b6688faa53747be7" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 36879 } } peers { permanent_uuid: "d537441aec13451f9c53abb19ee7f48e" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 33745 } }
I20260812 02:42:15.592984 2234 master_runner.cc:186] Error getting master registration for 127.7.132.189:36879: OK, instance_id { permanent_uuid: "23f2f837d4d046c5b6688faa53747be7" instance_seqno: 1786502534939325 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 02:42:15.614248 2234 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260812 02:42:15.623232 2263 tablet.cc:2369] T 00000000000000000000000000000000 P 104123a083ef4b9eba2f8b669d3d54c4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 02:42:15.623595 2539 sys_catalog.cc:455] T 00000000000000000000000000000000 P 104123a083ef4b9eba2f8b669d3d54c4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 104123a083ef4b9eba2f8b669d3d54c4. Latest consensus state: current_term: 1 leader_uuid: "104123a083ef4b9eba2f8b669d3d54c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "104123a083ef4b9eba2f8b669d3d54c4" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 44621 } } peers { permanent_uuid: "23f2f837d4d046c5b6688faa53747be7" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 36879 } } peers { permanent_uuid: "d537441aec13451f9c53abb19ee7f48e" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 33745 } } }
I20260812 02:42:15.624073 2539 sys_catalog.cc:458] T 00000000000000000000000000000000 P 104123a083ef4b9eba2f8b669d3d54c4 [sys.catalog]: This master's current role is: LEADER
I20260812 02:42:15.625524 2547 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:42:15.630249 2547 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 02:42:15.660476 2473 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d537441aec13451f9c53abb19ee7f48e [term 1 FOLLOWER]: Refusing update from remote peer 104123a083ef4b9eba2f8b669d3d54c4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:42:15.661846 2539 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 104123a083ef4b9eba2f8b669d3d54c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d537441aec13451f9c53abb19ee7f48e" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 33745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:42:15.671715 2413 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.132.188" port: 33745 } attempt: 5
Aug 12 02:42:15 dist-test-slave-mn4r krb5kdc[2216](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502534, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.189@KRBTEST.COM for kudu/127.7.132.188@KRBTEST.COM
I20260812 02:42:15.700681 2416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:15.672314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:33745 (local address 127.0.0.1:42236)
0812 02:42:15.672676 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:15.672695 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 02:42:15.672796 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:42:15.673622 (+ 826us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:42:15.673627 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:42:15.674186 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:42:15.674411 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:15.674424 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:15.675955 (+ 1531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:15.675961 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:15.677442 (+ 1481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:15.677459 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:15.678975 (+ 1516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:15.678981 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:15.679234 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:15.680157 (+ 923us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:42:15.680183 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:42:15.683863 (+ 3680us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:42:15.690299 (+ 6436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:42:15.690306 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:42:15.690332 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:42:15.691225 (+ 893us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:42:15.692052 (+ 827us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:42:15.692057 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:42:15.692061 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:42:15.692345 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:42:15.693576 (+ 1231us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:42:15.693582 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:42:15.693846 (+ 264us) client_negotiation.cc:769] Sending connection context
0812 02:42:15.694068 (+ 222us) client_negotiation.cc:240] Negotiation successful
0812 02:42:15.695311 (+ 1243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246}
I20260812 02:42:15.701429 2508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:15.673009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42236 (local address 127.7.132.188:33745)
0812 02:42:15.673178 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:15.673183 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:15.673201 (+ 18us) server_negotiation.cc:407] Connection header received
0812 02:42:15.673255 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:15.673260 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:15.673332 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:15.673442 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:15.674596 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:15.675734 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:15.677763 (+ 2029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:15.678570 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:15.678693 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:15.684176 (+ 5483us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:42:15.684183 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:42:15.684188 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:42:15.684231 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:42:15.690063 (+ 5832us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:15.691507 (+ 1444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:15.691513 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:15.691517 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:15.691842 (+ 325us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:15.692637 (+ 795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:15.692644 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:15.692647 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:15.693374 (+ 727us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:42:15.694988 (+ 1614us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:15.695718 (+ 730us) server_negotiation.cc:299] Negotiation successful
0812 02:42:15.699551 (+ 3833us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 02:42:15.708372 2413 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23f2f837d4d046c5b6688faa53747be7: Bootstrap starting.
I20260812 02:42:15.713100 2413 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 23f2f837d4d046c5b6688faa53747be7: Neither blocks nor log segments found. Creating new log.
W20260812 02:42:15.724814 2447 tablet.cc:2369] T 00000000000000000000000000000000 P d537441aec13451f9c53abb19ee7f48e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 02:42:15.732038 2420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:15.643937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55084 (local address 127.7.132.189:36879)
0812 02:42:15.644167 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:15.644175 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:15.670643 (+ 26468us) server_negotiation.cc:407] Connection header received
0812 02:42:15.671936 (+ 1293us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:15.671944 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:15.672043 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:15.672152 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:15.688002 (+ 15850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:15.689539 (+ 1537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:15.694311 (+ 4772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:15.695235 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:15.695360 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:15.716413 (+ 21053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:42:15.716423 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:42:15.716432 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:42:15.716496 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:42:15.721126 (+ 4630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:15.723701 (+ 2575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:15.723708 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:15.723711 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:15.723894 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:15.725699 (+ 1805us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:15.725705 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:15.725709 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:15.726159 (+ 450us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:42:15.726454 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:15.731373 (+ 4919us) server_negotiation.cc:299] Negotiation successful
0812 02:42:15.731658 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260812 02:42:15.738864 2413 log.cc:824] T 00000000000000000000000000000000 P 23f2f837d4d046c5b6688faa53747be7: Log is configured to *not* fsync() on all Append() calls
I20260812 02:42:15.743258 2512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:15.641917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42230 (local address 127.7.132.188:33745)
0812 02:42:15.643295 (+ 1378us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:15.643305 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:15.652130 (+ 8825us) server_negotiation.cc:407] Connection header received
0812 02:42:15.652215 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:15.652223 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:15.652356 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:15.652470 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:15.661924 (+ 9454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:15.663402 (+ 1478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:15.680847 (+ 17445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:15.681780 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:15.681926 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:15.727122 (+ 45196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:42:15.727133 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:42:15.727137 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:42:15.727180 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:42:15.731070 (+ 3890us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:15.739142 (+ 8072us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:15.739148 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:15.739152 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:15.739312 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:15.739931 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:15.739939 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:15.739945 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:15.740511 (+ 566us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:42:15.740853 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:15.742345 (+ 1492us) server_negotiation.cc:299] Negotiation successful
0812 02:42:15.742735 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1202}
I20260812 02:42:15.744256 2509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:15.643593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44902 (local address 127.7.132.190:44621)
0812 02:42:15.643767 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:15.643774 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:15.651492 (+ 7718us) server_negotiation.cc:407] Connection header received
0812 02:42:15.651733 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:15.651739 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:15.651837 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:15.651945 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:15.663719 (+ 11774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:15.665212 (+ 1493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:15.679263 (+ 14051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:15.680291 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:15.680423 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:15.733623 (+ 53200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:42:15.733631 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:42:15.733636 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:42:15.733684 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:42:15.737588 (+ 3904us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:15.738549 (+ 961us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:15.738557 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:15.738563 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:15.738780 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:15.741084 (+ 2304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:15.741089 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:15.741093 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:15.741517 (+ 424us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:42:15.741765 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:15.742930 (+ 1165us) server_negotiation.cc:299] Negotiation successful
0812 02:42:15.743198 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 02:42:15.749019 2537 sys_catalog.cc:455] T 00000000000000000000000000000000 P d537441aec13451f9c53abb19ee7f48e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 104123a083ef4b9eba2f8b669d3d54c4. Latest consensus state: current_term: 1 leader_uuid: "104123a083ef4b9eba2f8b669d3d54c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "104123a083ef4b9eba2f8b669d3d54c4" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 44621 } } peers { permanent_uuid: "23f2f837d4d046c5b6688faa53747be7" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 36879 } } peers { permanent_uuid: "d537441aec13451f9c53abb19ee7f48e" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 33745 } } }
I20260812 02:42:15.749351 2537 sys_catalog.cc:458] T 00000000000000000000000000000000 P d537441aec13451f9c53abb19ee7f48e [sys.catalog]: This master's current role is: FOLLOWER
I20260812 02:42:15.752272 2413 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23f2f837d4d046c5b6688faa53747be7: No bootstrap required, opened a new log
I20260812 02:42:15.757117 2552 mvcc.cc:205] Tried to move back new op lower bound from 7317514386040643584 to 7317514385874341888. Current Snapshot: MvccSnapshot[applied={T|T < 7317514386040643584}]
I20260812 02:42:15.766981 2413 raft_consensus.cc:348] T 00000000000000000000000000000000 P 23f2f837d4d046c5b6688faa53747be7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "104123a083ef4b9eba2f8b669d3d54c4" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 44621 } } peers { permanent_uuid: "23f2f837d4d046c5b6688faa53747be7" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 36879 } } peers { permanent_uuid: "d537441aec13451f9c53abb19ee7f48e" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 33745 } }
I20260812 02:42:15.767529 2413 raft_consensus.cc:729] T 00000000000000000000000000000000 P 23f2f837d4d046c5b6688faa53747be7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23f2f837d4d046c5b6688faa53747be7, State: Initialized, Role: FOLLOWER
I20260812 02:42:15.768163 2413 consensus_queue.cc:260] T 00000000000000000000000000000000 P 23f2f837d4d046c5b6688faa53747be7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "104123a083ef4b9eba2f8b669d3d54c4" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 44621 } } peers { permanent_uuid: "23f2f837d4d046c5b6688faa53747be7" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 36879 } } peers { permanent_uuid: "d537441aec13451f9c53abb19ee7f48e" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 33745 } }
I20260812 02:42:15.769474 2537 sys_catalog.cc:455] T 00000000000000000000000000000000 P d537441aec13451f9c53abb19ee7f48e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "104123a083ef4b9eba2f8b669d3d54c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "104123a083ef4b9eba2f8b669d3d54c4" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 44621 } } peers { permanent_uuid: "23f2f837d4d046c5b6688faa53747be7" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 36879 } } peers { permanent_uuid: "d537441aec13451f9c53abb19ee7f48e" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 33745 } } }
I20260812 02:42:15.769831 2537 sys_catalog.cc:458] T 00000000000000000000000000000000 P d537441aec13451f9c53abb19ee7f48e [sys.catalog]: This master's current role is: FOLLOWER
I20260812 02:42:15.778812 2539 sys_catalog.cc:455] T 00000000000000000000000000000000 P 104123a083ef4b9eba2f8b669d3d54c4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "104123a083ef4b9eba2f8b669d3d54c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "104123a083ef4b9eba2f8b669d3d54c4" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 44621 } } peers { permanent_uuid: "23f2f837d4d046c5b6688faa53747be7" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 36879 } } peers { permanent_uuid: "d537441aec13451f9c53abb19ee7f48e" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 33745 } } }
I20260812 02:42:15.779124 2539 sys_catalog.cc:458] T 00000000000000000000000000000000 P 104123a083ef4b9eba2f8b669d3d54c4 [sys.catalog]: This master's current role is: LEADER
I20260812 02:42:15.779294 2547 catalog_manager.cc:1347] Generated new cluster ID: a6bec04fb63a4d63b288c4c134938486
I20260812 02:42:15.779436 2547 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 02:42:15.795171 2565 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260812 02:42:15.796175 2568 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23f2f837d4d046c5b6688faa53747be7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "104123a083ef4b9eba2f8b669d3d54c4" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 44621 } } peers { permanent_uuid: "23f2f837d4d046c5b6688faa53747be7" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 36879 } } peers { permanent_uuid: "d537441aec13451f9c53abb19ee7f48e" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 33745 } } }
I20260812 02:42:15.801934 2568 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23f2f837d4d046c5b6688faa53747be7 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 02:42:15.807238 2547 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 02:42:15.809185 2547 catalog_manager.cc:1504] Loading token signing keys...
I20260812 02:42:15.802145 2424 master_runner.cc:186] Error getting master registration for 127.7.132.189:36879: OK, instance_id { permanent_uuid: "23f2f837d4d046c5b6688faa53747be7" instance_seqno: 1786502534939325 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 02:42:15.812467 2424 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 02:42:15.797605 2413 sys_catalog.cc:564] T 00000000000000000000000000000000 P 23f2f837d4d046c5b6688faa53747be7 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:42:15.798152 2380 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "104123a083ef4b9eba2f8b669d3d54c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "23f2f837d4d046c5b6688faa53747be7"
I20260812 02:42:15.798910 2379 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 23f2f837d4d046c5b6688faa53747be7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:42:15.817163 2379 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 23f2f837d4d046c5b6688faa53747be7 [term 1 FOLLOWER]: Refusing update from remote peer 104123a083ef4b9eba2f8b669d3d54c4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:42:15.824872 2380 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 23f2f837d4d046c5b6688faa53747be7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 104123a083ef4b9eba2f8b669d3d54c4 in term 1.
I20260812 02:42:15.828179 2539 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 104123a083ef4b9eba2f8b669d3d54c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "23f2f837d4d046c5b6688faa53747be7" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 36879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:42:15.836426 2547 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 104123a083ef4b9eba2f8b669d3d54c4: Generated new TSK 0
I20260812 02:42:15.837570 2547 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 02:42:15.873302 2568 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23f2f837d4d046c5b6688faa53747be7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 104123a083ef4b9eba2f8b669d3d54c4. Latest consensus state: current_term: 1 leader_uuid: "104123a083ef4b9eba2f8b669d3d54c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "104123a083ef4b9eba2f8b669d3d54c4" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 44621 } } peers { permanent_uuid: "23f2f837d4d046c5b6688faa53747be7" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 36879 } } peers { permanent_uuid: "d537441aec13451f9c53abb19ee7f48e" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 33745 } } }
I20260812 02:42:15.873283 2577 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23f2f837d4d046c5b6688faa53747be7 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "104123a083ef4b9eba2f8b669d3d54c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "104123a083ef4b9eba2f8b669d3d54c4" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 44621 } } peers { permanent_uuid: "23f2f837d4d046c5b6688faa53747be7" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 36879 } } peers { permanent_uuid: "d537441aec13451f9c53abb19ee7f48e" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 33745 } } }
I20260812 02:42:15.873698 2577 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23f2f837d4d046c5b6688faa53747be7 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 02:42:15.873699 2568 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23f2f837d4d046c5b6688faa53747be7 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 02:42:15.873840 2516 sys_catalog.cc:455] T 00000000000000000000000000000000 P 104123a083ef4b9eba2f8b669d3d54c4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "104123a083ef4b9eba2f8b669d3d54c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "104123a083ef4b9eba2f8b669d3d54c4" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 44621 } } peers { permanent_uuid: "23f2f837d4d046c5b6688faa53747be7" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 36879 } } peers { permanent_uuid: "d537441aec13451f9c53abb19ee7f48e" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 33745 } } }
I20260812 02:42:15.874228 2516 sys_catalog.cc:458] T 00000000000000000000000000000000 P 104123a083ef4b9eba2f8b669d3d54c4 [sys.catalog]: This master's current role is: LEADER
I20260812 02:42:15.897540 2593 catalog_manager.cc:1259] Loaded cluster ID: a6bec04fb63a4d63b288c4c134938486
I20260812 02:42:15.897745 2593 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 23f2f837d4d046c5b6688faa53747be7: loading cluster ID for follower catalog manager: success
I20260812 02:42:15.901716 2593 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 23f2f837d4d046c5b6688faa53747be7: acquiring CA information for follower catalog manager: success
I20260812 02:42:15.904405 2593 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 23f2f837d4d046c5b6688faa53747be7: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260812 02:42:15.989840 2509 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:42:15.971786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44912 (local address 127.7.132.190:44621)
0812 02:42:15.972013 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:15.972019 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:15.987717 (+ 15698us) server_negotiation.cc:407] Connection header received
0812 02:42:15.987967 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:15.987979 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:15.988092 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:15.988218 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:15.989571 (+ 1353us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44912: BlockingRecv error: recv got EOF from 127.0.0.1:44912 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
W20260812 02:42:15.990415 2512 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:42:15.978758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42240 (local address 127.7.132.188:33745)
0812 02:42:15.978945 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:15.978952 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:15.988579 (+ 9627us) server_negotiation.cc:407] Connection header received
0812 02:42:15.988850 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:15.988856 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:15.988945 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:15.989020 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:15.990152 (+ 1132us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42240: BlockingRecv error: recv got EOF from 127.0.0.1:42240 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
W20260812 02:42:15.993024 2420 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:42:15.973923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55086 (local address 127.7.132.189:36879)
0812 02:42:15.974143 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:15.974149 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:15.991402 (+ 17253us) server_negotiation.cc:407] Connection header received
0812 02:42:15.991571 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:15.991578 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:15.991661 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:15.991768 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:15.992813 (+ 1045us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55086: BlockingRecv error: recv got EOF from 127.0.0.1:55086 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
I20260812 02:42:16.006124 2420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:15.997648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55094 (local address 127.7.132.189:36879)
0812 02:42:15.997816 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:15.997822 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:15.998607 (+ 785us) server_negotiation.cc:407] Connection header received
0812 02:42:15.998822 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:15.998831 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:15.999045 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:15.999225 (+ 180us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 02:42:15.999709 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:16.000889 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:16.003124 (+ 2235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:16.003835 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:16.003971 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:16.004547 (+ 576us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 02:42:16.005310 (+ 763us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 02:42:16.005480 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:16.005597 (+ 117us) server_negotiation.cc:299] Negotiation successful
0812 02:42:16.005704 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
W20260812 02:42:16.024749 2512 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:42:16.016134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42256 (local address 127.7.132.188:33745)
0812 02:42:16.016324 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:16.016332 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:16.022747 (+ 6415us) server_negotiation.cc:407] Connection header received
0812 02:42:16.023330 (+ 583us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:16.023337 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:16.023467 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:16.023603 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:16.024486 (+ 883us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42256: BlockingRecv error: recv got EOF from 127.0.0.1:42256 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
I20260812 02:42:16.043830 2420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:16.036469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55096 (local address 127.7.132.189:36879)
0812 02:42:16.036707 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:16.036713 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:16.037294 (+ 581us) server_negotiation.cc:407] Connection header received
0812 02:42:16.037504 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:16.037512 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:16.037618 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:16.037719 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 02:42:16.038263 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:16.039485 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:16.041498 (+ 2013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:16.042305 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:16.042442 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:16.042962 (+ 520us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 02:42:16.043118 (+ 156us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 02:42:16.043221 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:16.043405 (+ 184us) server_negotiation.cc:299] Negotiation successful
0812 02:42:16.043493 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
W20260812 02:42:16.053998 2512 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:42:16.051602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42264 (local address 127.7.132.188:33745)
0812 02:42:16.051807 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:16.051813 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:16.052396 (+ 583us) server_negotiation.cc:407] Connection header received
0812 02:42:16.052596 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:16.052602 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:16.052697 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:16.052803 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:16.053749 (+ 946us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42264: BlockingRecv error: recv got EOF from 127.0.0.1:42264 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
I20260812 02:42:16.067035 2420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:16.059490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55104 (local address 127.7.132.189:36879)
0812 02:42:16.059686 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:16.059692 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:16.060237 (+ 545us) server_negotiation.cc:407] Connection header received
0812 02:42:16.060490 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:16.060496 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:16.060600 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:16.060696 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 02:42:16.061184 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:16.062430 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:16.064610 (+ 2180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:16.065306 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:16.065442 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:16.065990 (+ 548us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 02:42:16.066244 (+ 254us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 02:42:16.066322 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:16.066592 (+ 270us) server_negotiation.cc:299] Negotiation successful
0812 02:42:16.066687 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
W20260812 02:42:16.084353 2512 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:42:16.081572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42278 (local address 127.7.132.188:33745)
0812 02:42:16.081786 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:16.081793 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:16.082673 (+ 880us) server_negotiation.cc:407] Connection header received
0812 02:42:16.082897 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:16.082903 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:16.083004 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:16.083118 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:16.084103 (+ 985us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42278: BlockingRecv error: recv got EOF from 127.0.0.1:42278 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
I20260812 02:42:16.099834 2420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:16.092175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55120 (local address 127.7.132.189:36879)
0812 02:42:16.092376 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:16.092383 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:16.092976 (+ 593us) server_negotiation.cc:407] Connection header received
0812 02:42:16.093293 (+ 317us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:16.093300 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:16.093406 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:16.093476 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 02:42:16.094261 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:16.095444 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:16.097460 (+ 2016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:16.098264 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:16.098408 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:16.098939 (+ 531us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 02:42:16.099116 (+ 177us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 02:42:16.099235 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:16.099415 (+ 180us) server_negotiation.cc:299] Negotiation successful
0812 02:42:16.099516 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
W20260812 02:42:16.112341 2512 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:42:16.107870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42288 (local address 127.7.132.188:33745)
0812 02:42:16.108197 (+ 327us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:16.108204 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:16.110718 (+ 2514us) server_negotiation.cc:407] Connection header received
0812 02:42:16.110925 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:16.110931 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:16.111032 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:16.111145 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:16.112073 (+ 928us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42288: BlockingRecv error: recv got EOF from 127.0.0.1:42288 (error 108)
Metrics: {"server-negotiator.queue_time_us":175}
I20260812 02:42:16.131477 2420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:16.123995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55134 (local address 127.7.132.189:36879)
0812 02:42:16.124224 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:16.124230 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:16.124806 (+ 576us) server_negotiation.cc:407] Connection header received
0812 02:42:16.125209 (+ 403us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:16.125216 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:16.125334 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:16.125433 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 02:42:16.125987 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:16.127217 (+ 1230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:16.129252 (+ 2035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:16.129901 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:16.130034 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:16.130620 (+ 586us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 02:42:16.130770 (+ 150us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 02:42:16.130868 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:16.131047 (+ 179us) server_negotiation.cc:299] Negotiation successful
0812 02:42:16.131135 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
W20260812 02:42:16.141359 2512 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:42:16.139002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42302 (local address 127.7.132.188:33745)
0812 02:42:16.139199 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:16.139205 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:16.139761 (+ 556us) server_negotiation.cc:407] Connection header received
0812 02:42:16.139953 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:16.139959 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:16.140089 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:16.140213 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:16.141123 (+ 910us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42302: BlockingRecv error: recv got EOF from 127.0.0.1:42302 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 02:42:16.168646 2420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:16.161230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55148 (local address 127.7.132.189:36879)
0812 02:42:16.161437 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:16.161443 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:16.162029 (+ 586us) server_negotiation.cc:407] Connection header received
0812 02:42:16.162302 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:16.162308 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:16.162415 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:16.162519 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 02:42:16.163082 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:16.164346 (+ 1264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:16.166399 (+ 2053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:16.167129 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:16.167274 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:16.167799 (+ 525us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 02:42:16.167991 (+ 192us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 02:42:16.168092 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:16.168258 (+ 166us) server_negotiation.cc:299] Negotiation successful
0812 02:42:16.168364 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
W20260812 02:42:16.182868 2512 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:42:16.180260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42316 (local address 127.7.132.188:33745)
0812 02:42:16.180463 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:16.180470 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:16.181062 (+ 592us) server_negotiation.cc:407] Connection header received
0812 02:42:16.181389 (+ 327us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:16.181396 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:16.181518 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:16.181619 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:16.182611 (+ 992us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42316: BlockingRecv error: recv got EOF from 127.0.0.1:42316 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
I20260812 02:42:16.225737 2420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:16.218917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55158 (local address 127.7.132.189:36879)
0812 02:42:16.219172 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:16.219179 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:16.220179 (+ 1000us) server_negotiation.cc:407] Connection header received
0812 02:42:16.220325 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:16.220330 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:16.220439 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:16.220594 (+ 155us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 02:42:16.220988 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:16.222232 (+ 1244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:16.223691 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:16.224188 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:16.224348 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:16.224799 (+ 451us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 02:42:16.224908 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 02:42:16.225017 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:16.225169 (+ 152us) server_negotiation.cc:299] Negotiation successful
0812 02:42:16.225260 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
W20260812 02:42:16.235131 2512 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:42:16.232934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42330 (local address 127.7.132.188:33745)
0812 02:42:16.233158 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:16.233164 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:16.233502 (+ 338us) server_negotiation.cc:407] Connection header received
0812 02:42:16.233716 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:16.233720 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:16.233802 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:16.233886 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:16.234887 (+ 1001us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42330: BlockingRecv error: recv got EOF from 127.0.0.1:42330 (error 108)
Metrics: {"server-negotiator.queue_time_us":89}
I20260812 02:42:16.309427 2420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:16.303239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60616 (local address 127.7.132.189:36879)
0812 02:42:16.303464 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:16.303471 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:16.303864 (+ 393us) server_negotiation.cc:407] Connection header received
0812 02:42:16.304045 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:16.304051 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:16.304147 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:16.304274 (+ 127us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 02:42:16.304733 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:16.305924 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:16.307382 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:16.308105 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:16.308251 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:16.308614 (+ 363us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 02:42:16.308781 (+ 167us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 02:42:16.308874 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:16.309008 (+ 134us) server_negotiation.cc:299] Negotiation successful
0812 02:42:16.309095 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
W20260812 02:42:16.317910 2512 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:42:16.316002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34668 (local address 127.7.132.188:33745)
0812 02:42:16.316216 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:16.316222 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:16.316470 (+ 248us) server_negotiation.cc:407] Connection header received
0812 02:42:16.316637 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:16.316641 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:16.316714 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:16.316799 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:16.317685 (+ 886us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34668: BlockingRecv error: recv got EOF from 127.0.0.1:34668 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
I20260812 02:42:16.454543 2420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:16.449518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60632 (local address 127.7.132.189:36879)
0812 02:42:16.449711 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:16.449716 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:16.450029 (+ 313us) server_negotiation.cc:407] Connection header received
0812 02:42:16.450324 (+ 295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:16.450329 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:16.450406 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:16.450501 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 02:42:16.450909 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:16.451617 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:16.452899 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:16.453330 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:16.453486 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:16.453852 (+ 366us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 02:42:16.453962 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 02:42:16.454033 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:16.454233 (+ 200us) server_negotiation.cc:299] Negotiation successful
0812 02:42:16.454329 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
W20260812 02:42:16.462749 2512 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:42:16.460596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34682 (local address 127.7.132.188:33745)
0812 02:42:16.460798 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:16.460807 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:16.461237 (+ 430us) server_negotiation.cc:407] Connection header received
0812 02:42:16.461458 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:16.461462 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:16.461534 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:16.461643 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:16.462495 (+ 852us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34682: BlockingRecv error: recv got EOF from 127.0.0.1:34682 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 02:42:16.728947 2420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:16.723922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60646 (local address 127.7.132.189:36879)
0812 02:42:16.724127 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:16.724132 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:16.724356 (+ 224us) server_negotiation.cc:407] Connection header received
0812 02:42:16.724525 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:16.724529 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:16.724616 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:16.724757 (+ 141us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 02:42:16.725213 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:16.725987 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:16.727510 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:16.727899 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:16.727998 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:16.728358 (+ 360us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 02:42:16.728459 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 02:42:16.728525 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:16.728670 (+ 145us) server_negotiation.cc:299] Negotiation successful
0812 02:42:16.728741 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
W20260812 02:42:16.737006 2512 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:42:16.735050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34692 (local address 127.7.132.188:33745)
0812 02:42:16.735292 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:16.735298 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:16.735522 (+ 224us) server_negotiation.cc:407] Connection header received
0812 02:42:16.735737 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:16.735743 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:16.735827 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:16.735936 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:16.736739 (+ 803us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34692: BlockingRecv error: recv got EOF from 127.0.0.1:34692 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
I20260812 02:42:16.800602 2565 catalog_manager.cc:1259] Loaded cluster ID: a6bec04fb63a4d63b288c4c134938486
I20260812 02:42:16.800801 2565 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d537441aec13451f9c53abb19ee7f48e: loading cluster ID for follower catalog manager: success
I20260812 02:42:16.804114 2565 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d537441aec13451f9c53abb19ee7f48e: acquiring CA information for follower catalog manager: success
I20260812 02:42:16.806218 2565 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d537441aec13451f9c53abb19ee7f48e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 02:42:17.258365 2728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:17.252789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60660 (local address 127.7.132.189:36879)
0812 02:42:17.253291 (+ 502us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:17.253296 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:17.253318 (+ 22us) server_negotiation.cc:407] Connection header received
0812 02:42:17.253444 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:17.253448 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:17.253521 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:17.253640 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 02:42:17.254122 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:17.254991 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:17.256669 (+ 1678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:17.257086 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:17.257206 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:17.257583 (+ 377us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 02:42:17.257688 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 02:42:17.257758 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:17.257913 (+ 155us) server_negotiation.cc:299] Negotiation successful
0812 02:42:17.257991 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":399,"thread_start_us":276,"threads_started":1}
I20260812 02:42:17.271111 2738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:17.264413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34704 (local address 127.7.132.188:33745)
0812 02:42:17.264969 (+ 556us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:17.264974 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:17.265095 (+ 121us) server_negotiation.cc:407] Connection header received
0812 02:42:17.265265 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:17.265269 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:17.265427 (+ 158us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:17.265545 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 02:42:17.266018 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:17.267044 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:17.268589 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:17.269089 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:17.269207 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:17.269776 (+ 569us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 02:42:17.270474 (+ 698us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 02:42:17.270609 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:17.270751 (+ 142us) server_negotiation.cc:299] Negotiation successful
0812 02:42:17.270832 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":443,"thread_start_us":291,"threads_started":1}
I20260812 02:42:17.291363 2738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:17.280149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34720 (local address 127.7.132.188:33745)
0812 02:42:17.280367 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:17.280374 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:17.285502 (+ 5128us) server_negotiation.cc:407] Connection header received
0812 02:42:17.285829 (+ 327us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:17.285834 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:17.285932 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:17.286033 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 02:42:17.286742 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:17.287736 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:17.289366 (+ 1630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:17.289803 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:17.289943 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:17.290532 (+ 589us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 02:42:17.290672 (+ 140us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 02:42:17.290777 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:17.290976 (+ 199us) server_negotiation.cc:299] Negotiation successful
0812 02:42:17.291070 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260812 02:42:17.292312 2728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:17.280446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60672 (local address 127.7.132.189:36879)
0812 02:42:17.280666 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:17.280674 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:17.285230 (+ 4556us) server_negotiation.cc:407] Connection header received
0812 02:42:17.285511 (+ 281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:17.285519 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:17.285617 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:17.285713 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 02:42:17.286639 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:17.287400 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:17.290179 (+ 2779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:17.290707 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:17.290822 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:17.291501 (+ 679us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 02:42:17.291628 (+ 127us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 02:42:17.291717 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:17.291951 (+ 234us) server_negotiation.cc:299] Negotiation successful
0812 02:42:17.292039 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 02:42:17.293162 2744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:17.280149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44644 (local address 127.7.132.190:44621)
0812 02:42:17.284350 (+ 4201us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:17.284356 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:17.284692 (+ 336us) server_negotiation.cc:407] Connection header received
0812 02:42:17.285078 (+ 386us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:17.285086 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:17.285358 (+ 272us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:17.285555 (+ 197us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 02:42:17.286287 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:17.287437 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:17.290201 (+ 2764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:17.290720 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:17.290879 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:17.291677 (+ 798us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 02:42:17.292186 (+ 509us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 02:42:17.292302 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:17.292520 (+ 218us) server_negotiation.cc:299] Negotiation successful
0812 02:42:17.292879 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4066,"thread_start_us":3670,"threads_started":1}
I20260812 02:42:17.304747 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 2234
I20260812 02:42:17.306008 2315 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:42:17.426828 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 2234
W20260812 02:42:17.454039 2445 connection.cc:535] server connection from 127.0.0.1:42220 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 02:42:17.454697 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 2327
I20260812 02:42:17.455832 2407 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:42:17.573881 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 2327
I20260812 02:42:17.601413 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 2424
I20260812 02:42:17.602771 2498 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:42:17.710492 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 2424
2026-08-12T02:42:17Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (5440 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502156971023-7698-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 02:42:17 dist-test-slave-mn4r krb5kdc[2759](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 02:42:17 dist-test-slave-mn4r krb5kdc[2759](info): set up 2 sockets
Aug 12 02:42:17 dist-test-slave-mn4r krb5kdc[2759](info): commencing operation
krb5kdc: starting...
W20260812 02:42:18.966477 7698 mini_kdc.cc:121] Time spent starting KDC: real 1.208s user 0.003s sys 0.050s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502156971023-7698-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502156971023-7698-0/krb5kdc/test-user.keytab.
Aug 12 02:42:19 dist-test-slave-mn4r krb5kdc[2759](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502539, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T02:42:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T02:42:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.132.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
I20260812 02:42:19.330906 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502156971023-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502156971023-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502156971023-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:43121
--webserver_interface=127.7.132.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:43095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.132.190:43121,127.7.132.189:44363,127.7.132.188:34455
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 02:42:19.694679 2774 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 02:42:19.695385 2774 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:42:19.695538 2774 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:42:19.695704 2774 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:42:19.704967 2774 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:42:19.705116 2774 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:42:19.705248 2774 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:42:19.705338 2774 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:42:19.705451 2774 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:42:19.715874 2774 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:43095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502156971023-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502156971023-7698-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.132.190:43121,127.7.132.189:44363,127.7.132.188:34455
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:43121
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.132.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502156971023-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:42:19.717368 2774 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:42:19.718904 2774 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:42:19.731698 2788 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:42:19.731701 2782 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:42:19.731746 2783 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:42:19.732772 2774 server_base.cc:1034] running on GCE node
I20260812 02:42:19.733760 2774 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:42:19.735433 2774 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:42:19.736696 2774 hybrid_clock.cc:648] HybridClock initialized: now 1786502539736631 us; error 41 us; skew 500 ppm
I20260812 02:42:19.737057 2774 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:42:19 dist-test-slave-mn4r krb5kdc[2759](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502539, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:42:19.745220 2774 init.cc:377] Logged in from keytab as kudu/127.7.132.190@KRBTEST.COM (short username kudu)
I20260812 02:42:19.747452 2774 webserver.cc:460] Webserver started at http://127.7.132.190:45047/ using document root <none> and password file <none>
I20260812 02:42:19.748512 2774 fs_manager.cc:359] Metadata directory not provided
I20260812 02:42:19.748656 2774 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:42:19.749127 2774 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:42:19.753844 2774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502156971023-7698-0/minicluster-data/master-0/data/instance:
uuid: "da8eed8aa8f2485ea71b535405643c34"
format_stamp: "Formatted at 2026-08-12 02:42:19 on dist-test-slave-mn4r"
server_key: "6724f3350cbf921df0389f4e956b11af"
server_key_iv: "7c038447fab48ec69b201418e2817352"
server_key_version: "encryptionkey@0"
I20260812 02:42:19.755095 2774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502156971023-7698-0/minicluster-data/master-0/wal/instance:
uuid: "da8eed8aa8f2485ea71b535405643c34"
format_stamp: "Formatted at 2026-08-12 02:42:19 on dist-test-slave-mn4r"
server_key: "6724f3350cbf921df0389f4e956b11af"
server_key_iv: "7c038447fab48ec69b201418e2817352"
server_key_version: "encryptionkey@0"
I20260812 02:42:19.763010 2774 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.002s sys 0.005s
I20260812 02:42:19.768532 2795 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:42:19.770306 2774 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260812 02:42:19.770517 2774 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502156971023-7698-0/minicluster-data/master-0/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502156971023-7698-0/minicluster-data/master-0/wal
uuid: "da8eed8aa8f2485ea71b535405643c34"
format_stamp: "Formatted at 2026-08-12 02:42:19 on dist-test-slave-mn4r"
server_key: "6724f3350cbf921df0389f4e956b11af"
server_key_iv: "7c038447fab48ec69b201418e2817352"
server_key_version: "encryptionkey@0"
I20260812 02:42:19.770754 2774 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502156971023-7698-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502156971023-7698-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502156971023-7698-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:42:19.796237 2774 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:42:19.800793 2774 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:42:19.801113 2774 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:42:19.821465 2774 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.190:43121
I20260812 02:42:19.821511 2854 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.190:43121 every 8 connection(s)
I20260812 02:42:19.823017 2774 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502156971023-7698-0/minicluster-data/master-0/data/info.pb
I20260812 02:42:19.826979 2857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:42:19.830881 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 2774
I20260812 02:42:19.831102 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502156971023-7698-0/minicluster-data/master-0/wal/instance
I20260812 02:42:19.831574 7698 external_mini_cluster.cc:1428] Setting key 4d0ed91f2695b837da12b564bf413b85
I20260812 02:42:19.833922 2857 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.132.190" port: 43121 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 02:42:19 dist-test-slave-mn4r krb5kdc[2759](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502539, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.190@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 02:42:19.858644 2861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:19.837340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37470 (local address 127.7.132.190:43121)
0812 02:42:19.838299 (+ 959us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:19.838317 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:19.838904 (+ 587us) server_negotiation.cc:407] Connection header received
0812 02:42:19.841091 (+ 2187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:19.841103 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:19.841457 (+ 354us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:19.841703 (+ 246us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:19.843034 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:19.844968 (+ 1934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:19.846485 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:19.847090 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:19.847303 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:19.850844 (+ 3541us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:42:19.850853 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:42:19.850866 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:42:19.850896 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:42:19.853911 (+ 3015us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:19.854686 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:19.854693 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:19.854695 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:19.854842 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:19.855260 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:19.855270 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:19.855272 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:19.855784 (+ 512us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:42:19.856002 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:19.856568 (+ 566us) server_negotiation.cc:299] Negotiation successful
0812 02:42:19.857024 (+ 456us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":42,"server-negotiator.queue_time_us":657,"thread_start_us":401,"threads_started":1}
I20260812 02:42:19.858664 2862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:19.837340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:43121 (local address 127.0.0.1:37470)
0812 02:42:19.838703 (+ 1363us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:19.838755 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0812 02:42:19.840679 (+ 1924us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:42:19.841733 (+ 1054us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:42:19.841738 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:42:19.842438 (+ 700us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:42:19.842797 (+ 359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:19.842815 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:19.845138 (+ 2323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:19.845141 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:19.846335 (+ 1194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:19.846348 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:19.847352 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:19.847359 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:19.847528 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:19.848232 (+ 704us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:42:19.848255 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:42:19.850592 (+ 2337us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:42:19.854058 (+ 3466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:42:19.854071 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:42:19.854118 (+ 47us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:42:19.854524 (+ 406us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:42:19.854960 (+ 436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:42:19.854968 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:42:19.854973 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:42:19.855138 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:42:19.855936 (+ 798us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:42:19.855945 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:42:19.856180 (+ 235us) client_negotiation.cc:769] Sending connection context
0812 02:42:19.856427 (+ 247us) client_negotiation.cc:240] Negotiation successful
0812 02:42:19.857176 (+ 749us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1082,"mutex_wait_us":218,"spinlock_wait_cycles":70912,"thread_start_us":833,"threads_started":1}
I20260812 02:42:19.861658 2857 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.132.189" port: 44363 } has no permanent_uuid. Determining permanent_uuid...
I20260812 02:42:19.862854 2862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:19.862439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.189:44363 (local address 127.0.0.1:39148)
0812 02:42:19.862615 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:19.862739 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.189:44363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260812 02:42:19.863145 2798 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.132.189:44363: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.7.132.189@KRBTEST.COM; defaulting to no policy
I20260812 02:42:19.864828 2862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:19.864459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.189:44363 (local address 127.0.0.1:39150)
0812 02:42:19.864608 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:19.864684 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.189:44363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260812 02:42:19.865049 2857 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.132.189:44363: Network error: Client connection negotiation failed: client connection to 127.7.132.189:44363: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.189.keytab.
Entry for principal kudu/127.7.132.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.189.keytab.
I20260812 02:42:19.905848 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502156971023-7698-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502156971023-7698-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502156971023-7698-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502156971023-7698-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.189:44363
--webserver_interface=127.7.132.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.189.keytab
--principal=kudu/127.7.132.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:43095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.132.190:43121,127.7.132.189:44363,127.7.132.188:34455
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 02:42:19.914469 2857 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.132.189" port: 44363 } attempt: 1
I20260812 02:42:19.915753 2862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:19.915281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.189:44363 (local address 127.0.0.1:39152)
0812 02:42:19.915491 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:19.915620 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.189:44363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260812 02:42:19.916764 2862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:19.916398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.189:44363 (local address 127.0.0.1:39154)
0812 02:42:19.916560 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:19.916631 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.189:44363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260812 02:42:19.916949 2857 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.132.189:44363: Network error: Client connection negotiation failed: client connection to 127.7.132.189:44363: connect: Connection refused (error 111)
I20260812 02:42:19.985216 2857 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.132.189" port: 44363 } attempt: 2
I20260812 02:42:19.986624 2862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:19.985963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.189:44363 (local address 127.0.0.1:39158)
0812 02:42:19.986294 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:19.986425 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.189:44363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260812 02:42:19.987890 2862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:19.987449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.189:44363 (local address 127.0.0.1:39168)
0812 02:42:19.987628 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:19.987723 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.189:44363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260812 02:42:19.988155 2857 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.132.189:44363: Network error: Client connection negotiation failed: client connection to 127.7.132.189:44363: connect: Connection refused (error 111)
I20260812 02:42:20.079422 2857 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.132.189" port: 44363 } attempt: 3
I20260812 02:42:20.080783 2862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:20.080171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.189:44363 (local address 127.0.0.1:39182)
0812 02:42:20.080445 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:20.080609 (+ 164us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.189:44363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260812 02:42:20.081924 2862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:20.081485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.189:44363 (local address 127.0.0.1:39186)
0812 02:42:20.081637 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:20.081727 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.189:44363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260812 02:42:20.082180 2857 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.132.189:44363: Network error: Client connection negotiation failed: client connection to 127.7.132.189:44363: connect: Connection refused (error 111)
I20260812 02:42:20.225507 2857 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.132.189" port: 44363 } attempt: 4
I20260812 02:42:20.226990 2862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:20.226296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.189:44363 (local address 127.0.0.1:39190)
0812 02:42:20.226661 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:20.226788 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.189:44363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194}
I20260812 02:42:20.228125 2862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:20.227708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.189:44363 (local address 127.0.0.1:39202)
0812 02:42:20.227869 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:20.227975 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.189:44363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260812 02:42:20.228317 2857 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.132.189:44363: Network error: Client connection negotiation failed: client connection to 127.7.132.189:44363: connect: Connection refused (error 111)
W20260812 02:42:20.270361 2866 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 02:42:20.270705 2866 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:42:20.270771 2866 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:42:20.270855 2866 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:42:20.278522 2866 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:42:20.278623 2866 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:42:20.278681 2866 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:42:20.278733 2866 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:42:20.278792 2866 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:42:20.289029 2866 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:43095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502156971023-7698-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502156971023-7698-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.7.132.190:43121,127.7.132.189:44363,127.7.132.188:34455
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.189.keytab
--principal=kudu/127.7.132.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.189:44363
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502156971023-7698-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.7.132.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502156971023-7698-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:42:20.290549 2866 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:42:20.291975 2866 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:42:20.304490 2873 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:42:20.304453 2874 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:42:20.304546 2876 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:42:20.305541 2866 server_base.cc:1034] running on GCE node
I20260812 02:42:20.306522 2866 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:42:20.308084 2866 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:42:20.309386 2866 hybrid_clock.cc:648] HybridClock initialized: now 1786502540309295 us; error 56 us; skew 500 ppm
I20260812 02:42:20.309782 2866 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:42:20 dist-test-slave-mn4r krb5kdc[2759](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502540, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:42:20.313295 2866 init.cc:377] Logged in from keytab as kudu/127.7.132.189@KRBTEST.COM (short username kudu)
I20260812 02:42:20.314909 2866 webserver.cc:460] Webserver started at http://127.7.132.189:35941/ using document root <none> and password file <none>
I20260812 02:42:20.315632 2866 fs_manager.cc:359] Metadata directory not provided
I20260812 02:42:20.315768 2866 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:42:20.316108 2866 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:42:20.319279 2866 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502156971023-7698-0/minicluster-data/master-1/data/instance:
uuid: "866606d3e56d426586c079860654745d"
format_stamp: "Formatted at 2026-08-12 02:42:20 on dist-test-slave-mn4r"
server_key: "3b5cf5f4a92169a35e4e06a9998c6500"
server_key_iv: "f380ad3e5bbcf42683df21b64eea7590"
server_key_version: "encryptionkey@0"
I20260812 02:42:20.320115 2866 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502156971023-7698-0/minicluster-data/master-1/wal/instance:
uuid: "866606d3e56d426586c079860654745d"
format_stamp: "Formatted at 2026-08-12 02:42:20 on dist-test-slave-mn4r"
server_key: "3b5cf5f4a92169a35e4e06a9998c6500"
server_key_iv: "f380ad3e5bbcf42683df21b64eea7590"
server_key_version: "encryptionkey@0"
I20260812 02:42:20.325636 2866 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260812 02:42:20.329398 2884 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:42:20.331019 2866 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 02:42:20.331231 2866 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502156971023-7698-0/minicluster-data/master-1/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502156971023-7698-0/minicluster-data/master-1/wal
uuid: "866606d3e56d426586c079860654745d"
format_stamp: "Formatted at 2026-08-12 02:42:20 on dist-test-slave-mn4r"
server_key: "3b5cf5f4a92169a35e4e06a9998c6500"
server_key_iv: "f380ad3e5bbcf42683df21b64eea7590"
server_key_version: "encryptionkey@0"
I20260812 02:42:20.331478 2866 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502156971023-7698-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502156971023-7698-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502156971023-7698-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:42:20.346295 2866 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:42:20.351223 2866 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:42:20.351526 2866 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:42:20.370922 2866 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.189:44363
I20260812 02:42:20.370929 2944 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.189:44363 every 8 connection(s)
I20260812 02:42:20.372406 2866 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502156971023-7698-0/minicluster-data/master-1/data/info.pb
I20260812 02:42:20.376516 2945 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:42:20.381222 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 2866
I20260812 02:42:20.381551 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502156971023-7698-0/minicluster-data/master-1/wal/instance
I20260812 02:42:20.382144 7698 external_mini_cluster.cc:1428] Setting key 1176dfde830b438974642c83b3a64f2a
I20260812 02:42:20.384178 2945 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.132.190" port: 43121 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 02:42:20 dist-test-slave-mn4r krb5kdc[2759](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502540, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.189@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 02:42:20.406533 2947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:20.387268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37474 (local address 127.7.132.190:43121)
0812 02:42:20.387781 (+ 513us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:20.387786 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:20.389119 (+ 1333us) server_negotiation.cc:407] Connection header received
0812 02:42:20.391236 (+ 2117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:20.391243 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:20.391340 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:20.391472 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:20.393181 (+ 1709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:20.393911 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:20.395538 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:20.395956 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:20.396109 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:20.400291 (+ 4182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:42:20.400297 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:42:20.400300 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:42:20.400350 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:42:20.403250 (+ 2900us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:20.404057 (+ 807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:20.404062 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:20.404064 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:20.404188 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:20.404644 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:20.404648 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:20.404650 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:20.404953 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:42:20.405118 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:20.405868 (+ 750us) server_negotiation.cc:299] Negotiation successful
0812 02:42:20.406143 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":408,"thread_start_us":285,"threads_started":1}
I20260812 02:42:20.407588 2948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:20.388057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:43121 (local address 127.0.0.1:37474)
0812 02:42:20.388912 (+ 855us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:20.388954 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 02:42:20.390915 (+ 1961us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:42:20.391750 (+ 835us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:42:20.391759 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:42:20.392345 (+ 586us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:42:20.392941 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:20.392960 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:20.394121 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:20.394134 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:20.395370 (+ 1236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:20.395383 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:20.396225 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:20.396233 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:20.396487 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:20.397288 (+ 801us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:42:20.397331 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:42:20.400023 (+ 2692us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:42:20.403405 (+ 3382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:42:20.403414 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:42:20.403433 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:42:20.403905 (+ 472us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:42:20.404306 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:42:20.404311 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:42:20.404315 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:42:20.404509 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:42:20.405091 (+ 582us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:42:20.405099 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:42:20.405463 (+ 364us) client_negotiation.cc:769] Sending connection context
0812 02:42:20.405750 (+ 287us) client_negotiation.cc:240] Negotiation successful
0812 02:42:20.406337 (+ 587us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":594,"thread_start_us":378,"threads_started":1}
I20260812 02:42:20.409452 2945 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.132.189" port: 44363 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.7.132.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.188@KRBTEST.COM" created.
Aug 12 02:42:20 dist-test-slave-mn4r krb5kdc[2759](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502540, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.189@KRBTEST.COM for kudu/127.7.132.189@KRBTEST.COM
I20260812 02:42:20.426811 2948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:20.410377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.189:44363 (local address 127.0.0.1:39204)
0812 02:42:20.410586 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:20.410610 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0812 02:42:20.410732 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:42:20.411939 (+ 1207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:42:20.411942 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:42:20.412453 (+ 511us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:42:20.412607 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:20.412616 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:20.414962 (+ 2346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:20.414966 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:20.415817 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:20.415828 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:20.417160 (+ 1332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:20.417166 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:20.417344 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:20.417896 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:42:20.417913 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:42:20.420873 (+ 2960us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:42:20.424223 (+ 3350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:42:20.424232 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:42:20.424238 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:42:20.424610 (+ 372us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:42:20.424993 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:42:20.424998 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:42:20.425001 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:42:20.425149 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:42:20.425747 (+ 598us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:42:20.425753 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:42:20.425897 (+ 144us) client_negotiation.cc:769] Sending connection context
0812 02:42:20.426048 (+ 151us) client_negotiation.cc:240] Negotiation successful
0812 02:42:20.426386 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":71}
I20260812 02:42:20.426893 2951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:20.410546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39204 (local address 127.7.132.189:44363)
0812 02:42:20.411350 (+ 804us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:20.411361 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:20.411397 (+ 36us) server_negotiation.cc:407] Connection header received
0812 02:42:20.411459 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:20.411469 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:20.411758 (+ 289us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:20.411940 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:20.412813 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:20.414811 (+ 1998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:20.415972 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:20.416943 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:20.417086 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:20.421102 (+ 4016us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:42:20.421114 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:42:20.421128 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:42:20.421157 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:42:20.424081 (+ 2924us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:20.424751 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:20.424763 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:20.424765 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:20.424875 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:20.425252 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:20.425259 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:20.425262 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:20.425621 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:42:20.425843 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:20.426244 (+ 401us) server_negotiation.cc:299] Negotiation successful
0812 02:42:20.426512 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":25,"server-negotiator.queue_time_us":591,"thread_start_us":424,"threads_started":1}
I20260812 02:42:20.429337 2945 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.132.188" port: 34455 } has no permanent_uuid. Determining permanent_uuid...
I20260812 02:42:20.430512 2948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:20.430037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:34455 (local address 127.0.0.1:50954)
0812 02:42:20.430228 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:20.430344 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.188:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
W20260812 02:42:20.430763 2887 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.132.188:34455: connect: Connection refused (error 111)
I20260812 02:42:20.432581 2948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:20.432230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:34455 (local address 127.0.0.1:50964)
0812 02:42:20.432390 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:20.432469 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.188:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260812 02:42:20.432822 2945 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.132.188:34455: Network error: Client connection negotiation failed: client connection to 127.7.132.188:34455: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.188.keytab.
Entry for principal kudu/127.7.132.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.188.keytab.
I20260812 02:42:20.462920 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
/tmp/dist-test-taskSMpwqb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502156971023-7698-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502156971023-7698-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502156971023-7698-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502156971023-7698-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.188:34455
--webserver_interface=127.7.132.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.188.keytab
--principal=kudu/127.7.132.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:43095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.132.190:43121,127.7.132.189:44363,127.7.132.188:34455
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502156971023-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502156971023-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502156971023-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 02:42:20.482175 2945 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.132.188" port: 34455 } attempt: 1
I20260812 02:42:20.483515 2948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:20.482877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:34455 (local address 127.0.0.1:50968)
0812 02:42:20.483149 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:20.483295 (+ 146us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.188:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260812 02:42:20.484688 2948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:20.484210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:34455 (local address 127.0.0.1:50970)
0812 02:42:20.484422 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:20.484532 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.188:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260812 02:42:20.484921 2945 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.132.188:34455: Network error: Client connection negotiation failed: client connection to 127.7.132.188:34455: connect: Connection refused (error 111)
I20260812 02:42:20.527581 2857 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.132.189" port: 44363 } attempt: 5
Aug 12 02:42:20 dist-test-slave-mn4r krb5kdc[2759](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502539, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.190@KRBTEST.COM for kudu/127.7.132.189@KRBTEST.COM
I20260812 02:42:20.543601 2862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:20.528493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.189:44363 (local address 127.0.0.1:39218)
0812 02:42:20.528753 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:20.528786 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0812 02:42:20.528930 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:42:20.529360 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:42:20.529365 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:42:20.529804 (+ 439us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:42:20.530054 (+ 250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:20.530066 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:20.531552 (+ 1486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:20.531557 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:20.532661 (+ 1104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:20.532676 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:20.533691 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:20.533696 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:20.533882 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:20.534771 (+ 889us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:42:20.534823 (+ 52us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:42:20.536970 (+ 2147us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:42:20.540871 (+ 3901us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:42:20.540875 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:42:20.540878 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:42:20.541381 (+ 503us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:42:20.541855 (+ 474us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:42:20.541859 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:42:20.541862 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:42:20.542164 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:42:20.542804 (+ 640us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:42:20.542808 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:42:20.542959 (+ 151us) client_negotiation.cc:769] Sending connection context
0812 02:42:20.543102 (+ 143us) client_negotiation.cc:240] Negotiation successful
0812 02:42:20.543328 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":88}
I20260812 02:42:20.543823 2951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:20.528663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39218 (local address 127.7.132.189:44363)
0812 02:42:20.528902 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:20.528912 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:20.528941 (+ 29us) server_negotiation.cc:407] Connection header received
0812 02:42:20.529039 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:20.529047 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:20.529172 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:20.529314 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:20.530307 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:20.531385 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:20.532845 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:20.533514 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:20.533652 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:20.537188 (+ 3536us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:42:20.537195 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:42:20.537200 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:42:20.537242 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:42:20.540701 (+ 3459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:20.541526 (+ 825us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:20.541531 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:20.541535 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:20.541698 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:20.542286 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:20.542291 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:20.542294 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:20.542674 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:42:20.542832 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:20.543278 (+ 446us) server_negotiation.cc:299] Negotiation successful
0812 02:42:20.543520 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260812 02:42:20.544936 2857 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.132.188" port: 34455 } has no permanent_uuid. Determining permanent_uuid...
I20260812 02:42:20.546200 2862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:20.545647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:34455 (local address 127.0.0.1:50978)
0812 02:42:20.545845 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:20.545960 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.188:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260812 02:42:20.547248 2862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:20.546880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:34455 (local address 127.0.0.1:50986)
0812 02:42:20.547049 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:20.547144 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.188:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260812 02:42:20.547472 2857 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.132.188:34455: Network error: Client connection negotiation failed: client connection to 127.7.132.188:34455: connect: Connection refused (error 111)
I20260812 02:42:20.553170 2945 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.132.188" port: 34455 } attempt: 2
I20260812 02:42:20.554301 2948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:20.553813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:34455 (local address 127.0.0.1:51002)
0812 02:42:20.554001 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:20.554146 (+ 145us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.188:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260812 02:42:20.555256 2948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:20.554923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:34455 (local address 127.0.0.1:51004)
0812 02:42:20.555064 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:20.555141 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.188:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260812 02:42:20.555497 2945 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.132.188:34455: Network error: Client connection negotiation failed: client connection to 127.7.132.188:34455: connect: Connection refused (error 111)
I20260812 02:42:20.598724 2857 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.132.188" port: 34455 } attempt: 1
I20260812 02:42:20.600095 2862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:20.599554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:34455 (local address 127.0.0.1:51006)
0812 02:42:20.599776 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:20.599873 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.188:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260812 02:42:20.601122 2862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:20.600780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:34455 (local address 127.0.0.1:51010)
0812 02:42:20.600947 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:20.601015 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.188:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260812 02:42:20.601354 2857 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.132.188:34455: Network error: Client connection negotiation failed: client connection to 127.7.132.188:34455: connect: Connection refused (error 111)
I20260812 02:42:20.646763 2945 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.132.188" port: 34455 } attempt: 3
I20260812 02:42:20.648258 2948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:20.647504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:34455 (local address 127.0.0.1:51024)
0812 02:42:20.647827 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:20.648003 (+ 176us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.188:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101}
I20260812 02:42:20.649664 2948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:20.649064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:34455 (local address 127.0.0.1:51030)
0812 02:42:20.649350 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:20.649470 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.188:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151}
W20260812 02:42:20.649973 2945 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.132.188:34455: Network error: Client connection negotiation failed: client connection to 127.7.132.188:34455: connect: Connection refused (error 111)
I20260812 02:42:20.669584 2857 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.132.188" port: 34455 } attempt: 2
I20260812 02:42:20.670915 2862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:20.670369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:34455 (local address 127.0.0.1:51038)
0812 02:42:20.670633 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:20.670738 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.188:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260812 02:42:20.671981 2862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:20.671608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:34455 (local address 127.0.0.1:51054)
0812 02:42:20.671758 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:20.671862 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.188:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260812 02:42:20.672226 2857 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.132.188:34455: Network error: Client connection negotiation failed: client connection to 127.7.132.188:34455: connect: Connection refused (error 111)
I20260812 02:42:20.778465 2857 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.132.188" port: 34455 } attempt: 3
I20260812 02:42:20.779532 2862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:20.779063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:34455 (local address 127.0.0.1:51068)
0812 02:42:20.779275 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:20.779364 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.188:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260812 02:42:20.780522 2862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:20.780186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:34455 (local address 127.0.0.1:51076)
0812 02:42:20.780346 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:20.780424 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.188:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260812 02:42:20.780830 2857 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.132.188:34455: Network error: Client connection negotiation failed: client connection to 127.7.132.188:34455: connect: Connection refused (error 111)
I20260812 02:42:20.793237 2945 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.132.188" port: 34455 } attempt: 4
I20260812 02:42:20.794626 2948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:20.793992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:34455 (local address 127.0.0.1:51092)
0812 02:42:20.794292 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:20.794413 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.188:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
I20260812 02:42:20.795881 2948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:20.795474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:34455 (local address 127.0.0.1:51100)
0812 02:42:20.795645 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:20.795738 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.188:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260812 02:42:20.796128 2945 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.132.188:34455: Network error: Client connection negotiation failed: client connection to 127.7.132.188:34455: connect: Connection refused (error 111)
W20260812 02:42:20.827006 2956 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 02:42:20.827397 2956 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 02:42:20.827533 2956 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 02:42:20.827673 2956 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 02:42:20.835044 2956 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 02:42:20.835160 2956 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 02:42:20.835279 2956 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 02:42:20.835337 2956 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 02:42:20.835403 2956 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 02:42:20.845506 2956 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:43095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502156971023-7698-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502156971023-7698-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.7.132.190:43121,127.7.132.189:44363,127.7.132.188:34455
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502156971023-7698-0/krb5kdc/kudu_127.7.132.188.keytab
--principal=kudu/127.7.132.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.188:34455
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502156971023-7698-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.7.132.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502156971023-7698-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 01:43:53 UTC on ccycloud.quasar-wxbeky.root.comops.site
build id 4581825
ASAN enabled
I20260812 02:42:20.846951 2956 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 02:42:20.848333 2956 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 02:42:20.860595 2963 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:42:20.860692 2964 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 02:42:20.860641 2966 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 02:42:20.863660 2956 server_base.cc:1034] running on GCE node
I20260812 02:42:20.864435 2956 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 02:42:20.865839 2956 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 02:42:20.867079 2956 hybrid_clock.cc:648] HybridClock initialized: now 1786502540866986 us; error 58 us; skew 500 ppm
I20260812 02:42:20.867377 2956 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 02:42:20 dist-test-slave-mn4r krb5kdc[2759](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502540, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 02:42:20.870625 2956 init.cc:377] Logged in from keytab as kudu/127.7.132.188@KRBTEST.COM (short username kudu)
I20260812 02:42:20.872242 2956 webserver.cc:460] Webserver started at http://127.7.132.188:42723/ using document root <none> and password file <none>
I20260812 02:42:20.872992 2956 fs_manager.cc:359] Metadata directory not provided
I20260812 02:42:20.873128 2956 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 02:42:20.873469 2956 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 02:42:20.876654 2956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502156971023-7698-0/minicluster-data/master-2/data/instance:
uuid: "c430004289ea4516bc94ad66d9f69cb3"
format_stamp: "Formatted at 2026-08-12 02:42:20 on dist-test-slave-mn4r"
server_key: "ac87e3e0c2f96b62df8c0d9b9461effb"
server_key_iv: "b122ec8c1e73e88ae5eeabc879e2b031"
server_key_version: "encryptionkey@0"
I20260812 02:42:20.877489 2956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502156971023-7698-0/minicluster-data/master-2/wal/instance:
uuid: "c430004289ea4516bc94ad66d9f69cb3"
format_stamp: "Formatted at 2026-08-12 02:42:20 on dist-test-slave-mn4r"
server_key: "ac87e3e0c2f96b62df8c0d9b9461effb"
server_key_iv: "b122ec8c1e73e88ae5eeabc879e2b031"
server_key_version: "encryptionkey@0"
I20260812 02:42:20.883145 2956 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260812 02:42:20.887012 2976 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 02:42:20.888449 2956 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 02:42:20.888660 2956 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502156971023-7698-0/minicluster-data/master-2/data,/tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502156971023-7698-0/minicluster-data/master-2/wal
uuid: "c430004289ea4516bc94ad66d9f69cb3"
format_stamp: "Formatted at 2026-08-12 02:42:20 on dist-test-slave-mn4r"
server_key: "ac87e3e0c2f96b62df8c0d9b9461effb"
server_key_iv: "b122ec8c1e73e88ae5eeabc879e2b031"
server_key_version: "encryptionkey@0"
I20260812 02:42:20.888899 2956 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502156971023-7698-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502156971023-7698-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502156971023-7698-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 02:42:20.914525 2956 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 02:42:20.918802 2956 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 02:42:20.919108 2956 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 02:42:20.938381 2956 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.188:34455
I20260812 02:42:20.938395 3043 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.188:34455 every 8 connection(s)
I20260812 02:42:20.939899 2956 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502156971023-7698-0/minicluster-data/master-2/data/info.pb
I20260812 02:42:20.943924 3044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 02:42:20.949599 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu as pid 2956
I20260812 02:42:20.949821 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMpwqb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786502156971023-7698-0/minicluster-data/master-2/wal/instance
I20260812 02:42:20.950371 7698 external_mini_cluster.cc:1428] Setting key 86adc9cae8d34148f5a627b1be4bc5d1
I20260812 02:42:20.951370 3044 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.132.190" port: 43121 } has no permanent_uuid. Determining permanent_uuid...
I20260812 02:42:20.958117 2857 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.132.188" port: 34455 } attempt: 4
Aug 12 02:42:20 dist-test-slave-mn4r krb5kdc[2759](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502540, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.188@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
Aug 12 02:42:20 dist-test-slave-mn4r krb5kdc[2759](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502539, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.132.188@KRBTEST.COM
Aug 12 02:42:20 dist-test-slave-mn4r krb5kdc[2759](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502539, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.190@KRBTEST.COM for kudu/127.7.132.188@KRBTEST.COM
I20260812 02:42:20.975190 2862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:20.958751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:34455 (local address 127.0.0.1:51124)
0812 02:42:20.958946 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:20.958965 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 02:42:20.959060 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:42:20.959928 (+ 868us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:42:20.959932 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:42:20.960398 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:42:20.960534 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:20.960542 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:20.962121 (+ 1579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:20.962126 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:20.963372 (+ 1246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:20.963391 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:20.964163 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:20.964169 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:20.964341 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:20.964895 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:42:20.964912 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:42:20.967835 (+ 2923us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:42:20.972248 (+ 4413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:42:20.972253 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:42:20.972256 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:42:20.972713 (+ 457us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:42:20.973180 (+ 467us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:42:20.973184 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:42:20.973187 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:42:20.973338 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:42:20.974174 (+ 836us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:42:20.974179 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:42:20.974355 (+ 176us) client_negotiation.cc:769] Sending connection context
0812 02:42:20.974507 (+ 152us) client_negotiation.cc:240] Negotiation successful
0812 02:42:20.974769 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65}
I20260812 02:42:20.976105 3050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:20.954397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37490 (local address 127.7.132.190:43121)
0812 02:42:20.954895 (+ 498us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:20.954904 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:20.955397 (+ 493us) server_negotiation.cc:407] Connection header received
0812 02:42:20.956043 (+ 646us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:20.956049 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:20.956190 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:20.956311 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:20.957753 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:20.959494 (+ 1741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:20.961640 (+ 2146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:20.962126 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:20.962281 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:20.966769 (+ 4488us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:42:20.966774 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:42:20.966778 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:42:20.966816 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:42:20.970535 (+ 3719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:20.971750 (+ 1215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:20.971757 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:20.971762 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:20.971911 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:20.972470 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:20.972477 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:20.972480 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:20.972943 (+ 463us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:42:20.973110 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:20.975165 (+ 2055us) server_negotiation.cc:299] Negotiation successful
0812 02:42:20.975741 (+ 576us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":361,"thread_start_us":264,"threads_started":1}
I20260812 02:42:20.976197 3049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:20.952358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51114 (local address 127.7.132.188:34455)
0812 02:42:20.953211 (+ 853us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:20.953223 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:20.953267 (+ 44us) server_negotiation.cc:407] Connection header received
0812 02:42:20.955774 (+ 2507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:20.955795 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:20.956042 (+ 247us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:20.956214 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:20.957658 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:20.959389 (+ 1731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:20.961207 (+ 1818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:20.962227 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:20.962991 (+ 764us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:20.967724 (+ 4733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:42:20.967738 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:42:20.967755 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:42:20.967793 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:42:20.971932 (+ 4139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:20.972881 (+ 949us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:20.972886 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:20.972890 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:20.973065 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:20.973536 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:20.973539 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:20.973541 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:20.973856 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:42:20.974217 (+ 361us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:20.974555 (+ 338us) server_negotiation.cc:299] Negotiation successful
0812 02:42:20.974811 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":45,"server-negotiator.queue_time_us":647,"thread_start_us":388,"threads_started":1}
I20260812 02:42:20.976163 3051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:20.954520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:43121 (local address 127.0.0.1:37490)
0812 02:42:20.955249 (+ 729us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:20.955286 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 02:42:20.955786 (+ 500us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:42:20.956409 (+ 623us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:42:20.956416 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:42:20.957172 (+ 756us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:42:20.957543 (+ 371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:20.957592 (+ 49us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:20.959658 (+ 2066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:20.959664 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:20.961476 (+ 1812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:20.961489 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:20.962350 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:20.962358 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:20.962593 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:20.963510 (+ 917us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:42:20.963553 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:42:20.966576 (+ 3023us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:42:20.970681 (+ 4105us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:42:20.970691 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:42:20.970704 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:42:20.971587 (+ 883us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:42:20.972055 (+ 468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:42:20.972071 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:42:20.972075 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:42:20.972336 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:42:20.973107 (+ 771us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:42:20.973118 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:42:20.973596 (+ 478us) client_negotiation.cc:769] Sending connection context
0812 02:42:20.973979 (+ 383us) client_negotiation.cc:240] Negotiation successful
0812 02:42:20.974583 (+ 604us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":522,"spinlock_wait_cycles":22784,"thread_start_us":376,"threads_started":1}
I20260812 02:42:20.976414 3052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:20.958913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51124 (local address 127.7.132.188:34455)
0812 02:42:20.959664 (+ 751us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:20.959668 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:20.959685 (+ 17us) server_negotiation.cc:407] Connection header received
0812 02:42:20.959723 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:20.959726 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:20.959781 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:20.959905 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:20.960707 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:20.961957 (+ 1250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:20.963525 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:20.963958 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:20.964146 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:20.968129 (+ 3983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:42:20.968136 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:42:20.968140 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:42:20.968177 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:42:20.972117 (+ 3940us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:20.972851 (+ 734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:20.972860 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:20.972863 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:20.973007 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:20.973461 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:20.973465 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:20.973469 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:20.973911 (+ 442us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:42:20.974102 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:20.975639 (+ 1537us) server_negotiation.cc:299] Negotiation successful
0812 02:42:20.975861 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":40,"server-negotiator.queue_time_us":596,"thread_start_us":387,"threads_started":1}
I20260812 02:42:20.978166 3044 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.132.189" port: 44363 } has no permanent_uuid. Determining permanent_uuid...
I20260812 02:42:20.984866 2857 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da8eed8aa8f2485ea71b535405643c34: Bootstrap starting.
Aug 12 02:42:20 dist-test-slave-mn4r krb5kdc[2759](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502540, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.188@KRBTEST.COM for kudu/127.7.132.189@KRBTEST.COM
Aug 12 02:42:20 dist-test-slave-mn4r krb5kdc[2759](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502539, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 02:42:20.989388 2857 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P da8eed8aa8f2485ea71b535405643c34: Neither blocks nor log segments found. Creating new log.
I20260812 02:42:20.990860 2857 log.cc:824] T 00000000000000000000000000000000 P da8eed8aa8f2485ea71b535405643c34: Log is configured to *not* fsync() on all Append() calls
I20260812 02:42:20.994266 3051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:20.978870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.189:44363 (local address 127.0.0.1:39230)
0812 02:42:20.979060 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:20.979080 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 02:42:20.979191 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:42:20.979542 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:42:20.979546 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:42:20.980011 (+ 465us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:42:20.980574 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:20.980591 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:20.982103 (+ 1512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:20.982108 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:20.983415 (+ 1307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:20.983432 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:20.984526 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:20.984531 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:20.984663 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:20.985730 (+ 1067us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:42:20.985760 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:42:20.987920 (+ 2160us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:42:20.991445 (+ 3525us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:42:20.991452 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:42:20.991456 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:42:20.991924 (+ 468us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:42:20.992384 (+ 460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:42:20.992388 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:42:20.992390 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:42:20.992570 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:42:20.993175 (+ 605us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:42:20.993180 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:42:20.993362 (+ 182us) client_negotiation.cc:769] Sending connection context
0812 02:42:20.993547 (+ 185us) client_negotiation.cc:240] Negotiation successful
0812 02:42:20.993846 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64}
I20260812 02:42:20.994637 2951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:20.978947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39230 (local address 127.7.132.189:44363)
0812 02:42:20.979153 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:20.979159 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:20.979182 (+ 23us) server_negotiation.cc:407] Connection header received
0812 02:42:20.979327 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:20.979333 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:20.979414 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:20.979519 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:20.980778 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:20.981867 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:20.983574 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:20.984213 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:20.984334 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:20.988125 (+ 3791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:42:20.988129 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:42:20.988133 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:42:20.988168 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:42:20.991314 (+ 3146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:20.992056 (+ 742us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:20.992064 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:20.992069 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:20.992226 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:20.992690 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:20.992696 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:20.992702 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:20.993050 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:42:20.993305 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:20.993709 (+ 404us) server_negotiation.cc:299] Negotiation successful
0812 02:42:20.993950 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 02:42:20.995538 3044 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.132.188" port: 34455 } has no permanent_uuid. Determining permanent_uuid...
I20260812 02:42:20.998404 2857 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da8eed8aa8f2485ea71b535405643c34: No bootstrap required, opened a new log
I20260812 02:42:20.998410 3050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:20.980313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37492 (local address 127.7.132.190:43121)
0812 02:42:20.980509 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:20.980514 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:20.980532 (+ 18us) server_negotiation.cc:407] Connection header received
0812 02:42:20.980581 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:20.980586 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:20.980671 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:20.980771 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:20.982048 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:20.982819 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:20.984335 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:20.985007 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:20.985183 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:20.990356 (+ 5173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:42:20.990363 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:42:20.990367 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:42:20.990406 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:42:20.995255 (+ 4849us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:20.996148 (+ 893us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:20.996154 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:20.996157 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:20.996323 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:20.996823 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:20.996830 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:20.996833 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:20.997147 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:42:20.997472 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:20.997789 (+ 317us) server_negotiation.cc:299] Negotiation successful
0812 02:42:20.998060 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
Aug 12 02:42:20 dist-test-slave-mn4r krb5kdc[2759](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502540, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.188@KRBTEST.COM for kudu/127.7.132.188@KRBTEST.COM
Aug 12 02:42:21 dist-test-slave-mn4r krb5kdc[2759](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502539, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.132.189@KRBTEST.COM
I20260812 02:42:21.009591 2857 raft_consensus.cc:348] T 00000000000000000000000000000000 P da8eed8aa8f2485ea71b535405643c34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da8eed8aa8f2485ea71b535405643c34" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 43121 } } peers { permanent_uuid: "866606d3e56d426586c079860654745d" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 44363 } } peers { permanent_uuid: "c430004289ea4516bc94ad66d9f69cb3" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 34455 } }
I20260812 02:42:21.010068 2857 raft_consensus.cc:729] T 00000000000000000000000000000000 P da8eed8aa8f2485ea71b535405643c34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da8eed8aa8f2485ea71b535405643c34, State: Initialized, Role: FOLLOWER
I20260812 02:42:21.010660 2857 consensus_queue.cc:260] T 00000000000000000000000000000000 P da8eed8aa8f2485ea71b535405643c34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da8eed8aa8f2485ea71b535405643c34" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 43121 } } peers { permanent_uuid: "866606d3e56d426586c079860654745d" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 44363 } } peers { permanent_uuid: "c430004289ea4516bc94ad66d9f69cb3" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 34455 } }
I20260812 02:42:21.011772 3054 sys_catalog.cc:455] T 00000000000000000000000000000000 P da8eed8aa8f2485ea71b535405643c34 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da8eed8aa8f2485ea71b535405643c34" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 43121 } } peers { permanent_uuid: "866606d3e56d426586c079860654745d" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 44363 } } peers { permanent_uuid: "c430004289ea4516bc94ad66d9f69cb3" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 34455 } } }
I20260812 02:42:21.012342 3054 sys_catalog.cc:458] T 00000000000000000000000000000000 P da8eed8aa8f2485ea71b535405643c34 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 02:42:21.012418 3051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:20.996334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:34455 (local address 127.0.0.1:51130)
0812 02:42:20.996576 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:20.996596 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 02:42:20.996707 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:42:20.997863 (+ 1156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:42:20.997867 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:42:20.998387 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:42:20.998613 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:20.998624 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:20.999673 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:20.999676 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:21.000677 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:21.000694 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:21.001756 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:21.001765 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:21.002025 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:21.002684 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:42:21.002710 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:42:21.004988 (+ 2278us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:42:21.009011 (+ 4023us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:42:21.009019 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:42:21.009022 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:42:21.009568 (+ 546us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:42:21.010331 (+ 763us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:42:21.010336 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:42:21.010339 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:42:21.010516 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:42:21.011364 (+ 848us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:42:21.011369 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:42:21.011522 (+ 153us) client_negotiation.cc:769] Sending connection context
0812 02:42:21.011699 (+ 177us) client_negotiation.cc:240] Negotiation successful
0812 02:42:21.012012 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130}
I20260812 02:42:21.013279 2857 sys_catalog.cc:564] T 00000000000000000000000000000000 P da8eed8aa8f2485ea71b535405643c34 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:42:21.013331 3049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:20.996416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51130 (local address 127.7.132.188:34455)
0812 02:42:20.996587 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:20.996593 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:20.996650 (+ 57us) server_negotiation.cc:407] Connection header received
0812 02:42:20.997555 (+ 905us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:20.997561 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:20.997641 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:20.997727 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:20.998744 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:20.999532 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:21.000814 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:21.001410 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:21.001523 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:21.005206 (+ 3683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:42:21.005213 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:42:21.005218 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:42:21.005258 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:42:21.008852 (+ 3594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:21.009706 (+ 854us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:21.009712 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:21.009715 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:21.009966 (+ 251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:21.010652 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:21.010659 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:21.010662 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:21.011114 (+ 452us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:42:21.011305 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:21.012806 (+ 1501us) server_negotiation.cc:299] Negotiation successful
0812 02:42:21.013028 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 02:42:21.018726 2951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:21.000579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39240 (local address 127.7.132.189:44363)
0812 02:42:21.000779 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:21.000785 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:21.000807 (+ 22us) server_negotiation.cc:407] Connection header received
0812 02:42:21.000886 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:21.000892 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:21.000965 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:21.001074 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:21.002762 (+ 1688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:21.003860 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:21.005568 (+ 1708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:21.006200 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:21.006326 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:21.010935 (+ 4609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:42:21.010944 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:42:21.010949 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:42:21.010994 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:42:21.014739 (+ 3745us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:21.015633 (+ 894us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:21.015639 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:21.015645 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:21.015799 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:21.016274 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:21.016279 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:21.016284 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:21.016639 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:42:21.017970 (+ 1331us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:21.018147 (+ 177us) server_negotiation.cc:299] Negotiation successful
0812 02:42:21.018388 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260812 02:42:21.019631 3044 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c430004289ea4516bc94ad66d9f69cb3: Bootstrap starting.
I20260812 02:42:21.023792 3044 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c430004289ea4516bc94ad66d9f69cb3: Neither blocks nor log segments found. Creating new log.
I20260812 02:42:21.023952 7698 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 02:42:21.025218 3044 log.cc:824] T 00000000000000000000000000000000 P c430004289ea4516bc94ad66d9f69cb3: Log is configured to *not* fsync() on all Append() calls
I20260812 02:42:21.028062 2825 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "da8eed8aa8f2485ea71b535405643c34"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37492
I20260812 02:42:21.028558 2825 raft_consensus.cc:482] T 00000000000000000000000000000000 P da8eed8aa8f2485ea71b535405643c34 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 02:42:21.028834 2825 raft_consensus.cc:3037] T 00000000000000000000000000000000 P da8eed8aa8f2485ea71b535405643c34 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:42:21.032229 2825 raft_consensus.cc:504] T 00000000000000000000000000000000 P da8eed8aa8f2485ea71b535405643c34 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da8eed8aa8f2485ea71b535405643c34" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 43121 } } peers { permanent_uuid: "866606d3e56d426586c079860654745d" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 44363 } } peers { permanent_uuid: "c430004289ea4516bc94ad66d9f69cb3" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 34455 } }
I20260812 02:42:21.033598 2825 leader_election.cc:288] T 00000000000000000000000000000000 P da8eed8aa8f2485ea71b535405643c34 [CANDIDATE]: Term 1 election: Requested vote from peers 866606d3e56d426586c079860654745d (127.7.132.189:44363), c430004289ea4516bc94ad66d9f69cb3 (127.7.132.188:34455)
I20260812 02:42:21.023322 2774 master_runner.cc:186] Error getting master registration for 127.7.132.189:44363: OK, instance_id { permanent_uuid: "866606d3e56d426586c079860654745d" instance_seqno: 1786502540365260 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 02:42:21.038612 2774 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 02:42:21.039875 3044 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c430004289ea4516bc94ad66d9f69cb3: No bootstrap required, opened a new log
W20260812 02:42:21.044786 3067 catalog_manager.cc:1558] T 00000000000000000000000000000000 P da8eed8aa8f2485ea71b535405643c34: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 02:42:21.044916 3067 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260812 02:42:21.055251 2802 tablet.cc:2369] T 00000000000000000000000000000000 P da8eed8aa8f2485ea71b535405643c34: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 02:42:21.057462 3044 raft_consensus.cc:348] T 00000000000000000000000000000000 P c430004289ea4516bc94ad66d9f69cb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da8eed8aa8f2485ea71b535405643c34" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 43121 } } peers { permanent_uuid: "866606d3e56d426586c079860654745d" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 44363 } } peers { permanent_uuid: "c430004289ea4516bc94ad66d9f69cb3" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 34455 } }
I20260812 02:42:21.058112 3044 raft_consensus.cc:729] T 00000000000000000000000000000000 P c430004289ea4516bc94ad66d9f69cb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c430004289ea4516bc94ad66d9f69cb3, State: Initialized, Role: FOLLOWER
I20260812 02:42:21.058883 3044 consensus_queue.cc:260] T 00000000000000000000000000000000 P c430004289ea4516bc94ad66d9f69cb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da8eed8aa8f2485ea71b535405643c34" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 43121 } } peers { permanent_uuid: "866606d3e56d426586c079860654745d" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 44363 } } peers { permanent_uuid: "c430004289ea4516bc94ad66d9f69cb3" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 34455 } }
I20260812 02:42:21.060575 3076 sys_catalog.cc:455] T 00000000000000000000000000000000 P c430004289ea4516bc94ad66d9f69cb3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da8eed8aa8f2485ea71b535405643c34" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 43121 } } peers { permanent_uuid: "866606d3e56d426586c079860654745d" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 44363 } } peers { permanent_uuid: "c430004289ea4516bc94ad66d9f69cb3" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 34455 } } }
I20260812 02:42:21.061632 3076 sys_catalog.cc:458] T 00000000000000000000000000000000 P c430004289ea4516bc94ad66d9f69cb3 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 02:42:21.062212 3044 sys_catalog.cc:564] T 00000000000000000000000000000000 P c430004289ea4516bc94ad66d9f69cb3 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:42:21.062259 3011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "da8eed8aa8f2485ea71b535405643c34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "c430004289ea4516bc94ad66d9f69cb3"
I20260812 02:42:21.062783 3011 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c430004289ea4516bc94ad66d9f69cb3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:42:21.066543 3011 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c430004289ea4516bc94ad66d9f69cb3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da8eed8aa8f2485ea71b535405643c34 in term 1.
I20260812 02:42:21.067346 2798 leader_election.cc:302] T 00000000000000000000000000000000 P da8eed8aa8f2485ea71b535405643c34 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c430004289ea4516bc94ad66d9f69cb3, da8eed8aa8f2485ea71b535405643c34; no voters:
I20260812 02:42:21.067921 3054 raft_consensus.cc:2781] T 00000000000000000000000000000000 P da8eed8aa8f2485ea71b535405643c34 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 02:42:21.068598 3054 raft_consensus.cc:686] T 00000000000000000000000000000000 P da8eed8aa8f2485ea71b535405643c34 [term 1 LEADER]: Becoming Leader. State: Replica: da8eed8aa8f2485ea71b535405643c34, State: Running, Role: LEADER
I20260812 02:42:21.069253 3054 consensus_queue.cc:237] T 00000000000000000000000000000000 P da8eed8aa8f2485ea71b535405643c34 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da8eed8aa8f2485ea71b535405643c34" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 43121 } } peers { permanent_uuid: "866606d3e56d426586c079860654745d" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 44363 } } peers { permanent_uuid: "c430004289ea4516bc94ad66d9f69cb3" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 34455 } }
I20260812 02:42:21.084369 3081 sys_catalog.cc:455] T 00000000000000000000000000000000 P da8eed8aa8f2485ea71b535405643c34 [sys.catalog]: SysCatalogTable state changed. Reason: New leader da8eed8aa8f2485ea71b535405643c34. Latest consensus state: current_term: 1 leader_uuid: "da8eed8aa8f2485ea71b535405643c34" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da8eed8aa8f2485ea71b535405643c34" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 43121 } } peers { permanent_uuid: "866606d3e56d426586c079860654745d" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 44363 } } peers { permanent_uuid: "c430004289ea4516bc94ad66d9f69cb3" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 34455 } } }
I20260812 02:42:21.084846 3081 sys_catalog.cc:458] T 00000000000000000000000000000000 P da8eed8aa8f2485ea71b535405643c34 [sys.catalog]: This master's current role is: LEADER
I20260812 02:42:21.092828 3088 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 02:42:21.095335 2945 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.132.188" port: 34455 } attempt: 5
I20260812 02:42:21.095844 3088 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 02:42:21.106431 3049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:21.071016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51138 (local address 127.7.132.188:34455)
0812 02:42:21.071213 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:21.071219 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:21.081842 (+ 10623us) server_negotiation.cc:407] Connection header received
0812 02:42:21.082358 (+ 516us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:21.082364 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:21.082482 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:21.082561 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:21.084974 (+ 2413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:21.086349 (+ 1375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:21.091033 (+ 4684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:21.092005 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:21.092147 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:21.098625 (+ 6478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:42:21.098635 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:42:21.098639 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:42:21.098683 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:42:21.102760 (+ 4077us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:21.103615 (+ 855us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:21.103624 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:21.103630 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:21.103812 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:21.104312 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:21.104317 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:21.104321 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:21.104698 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:42:21.104945 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:21.105603 (+ 658us) server_negotiation.cc:299] Negotiation successful
0812 02:42:21.105922 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
Aug 12 02:42:21 dist-test-slave-mn4r krb5kdc[2759](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786502540, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.189@KRBTEST.COM for kudu/127.7.132.188@KRBTEST.COM
I20260812 02:42:21.115594 3011 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c430004289ea4516bc94ad66d9f69cb3 [term 1 FOLLOWER]: Refusing update from remote peer da8eed8aa8f2485ea71b535405643c34: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:42:21.118717 3054 consensus_queue.cc:1035] T 00000000000000000000000000000000 P da8eed8aa8f2485ea71b535405643c34 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c430004289ea4516bc94ad66d9f69cb3" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 34455 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 02:42:21.124598 2948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:21.095912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:34455 (local address 127.0.0.1:51152)
0812 02:42:21.096292 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0812 02:42:21.096312 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 02:42:21.096416 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 02:42:21.097440 (+ 1024us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 02:42:21.097446 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 02:42:21.097941 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0812 02:42:21.098193 (+ 252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:21.098207 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:21.099809 (+ 1602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:21.099815 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:21.101262 (+ 1447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 02:42:21.101278 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 02:42:21.102953 (+ 1675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 02:42:21.102959 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 02:42:21.103227 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:21.104105 (+ 878us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 02:42:21.104138 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0812 02:42:21.108477 (+ 4339us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 02:42:21.116788 (+ 8311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:42:21.116795 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:42:21.116800 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:42:21.117785 (+ 985us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:42:21.118546 (+ 761us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 02:42:21.118574 (+ 28us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 02:42:21.118577 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 02:42:21.118919 (+ 342us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 02:42:21.120210 (+ 1291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 02:42:21.120216 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 02:42:21.120482 (+ 266us) client_negotiation.cc:769] Sending connection context
0812 02:42:21.120736 (+ 254us) client_negotiation.cc:240] Negotiation successful
0812 02:42:21.122101 (+ 1365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248}
I20260812 02:42:21.130415 2951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:21.069226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39250 (local address 127.7.132.189:44363)
0812 02:42:21.069425 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:21.069432 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:21.071043 (+ 1611us) server_negotiation.cc:407] Connection header received
0812 02:42:21.071500 (+ 457us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:21.071507 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:21.071608 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:21.071688 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:21.075838 (+ 4150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:21.076988 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:21.080486 (+ 3498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:21.081603 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:21.081717 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:21.087760 (+ 6043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:42:21.087770 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:42:21.087777 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:42:21.087823 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:42:21.091855 (+ 4032us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:21.093519 (+ 1664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:21.093525 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:21.093528 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:21.093708 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:21.094459 (+ 751us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:21.094465 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:21.094468 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:21.094914 (+ 446us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:42:21.095185 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:21.125219 (+ 30034us) server_negotiation.cc:299] Negotiation successful
0812 02:42:21.130007 (+ 4788us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260812 02:42:21.135324 3050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:21.073550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37498 (local address 127.7.132.190:43121)
0812 02:42:21.073764 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:21.073772 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:21.098654 (+ 24882us) server_negotiation.cc:407] Connection header received
0812 02:42:21.098923 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:21.098929 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:21.099029 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:21.099103 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:21.102260 (+ 3157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:21.103469 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:21.109919 (+ 6450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:21.110679 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:21.110787 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:21.117087 (+ 6300us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:42:21.117097 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:42:21.117105 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:42:21.117167 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:42:21.128962 (+ 11795us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:21.130687 (+ 1725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:21.130693 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:21.130697 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:21.130861 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:21.131421 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:21.131426 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:21.131430 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:21.131842 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:42:21.134450 (+ 2608us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:21.134588 (+ 138us) server_negotiation.cc:299] Negotiation successful
0812 02:42:21.134848 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 02:42:21.137221 3052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:21.096629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51152 (local address 127.7.132.188:34455)
0812 02:42:21.096831 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:21.096837 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:21.096856 (+ 19us) server_negotiation.cc:407] Connection header received
0812 02:42:21.096910 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:21.096915 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:21.096992 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:21.097104 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:21.098411 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:21.099597 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:21.101815 (+ 2218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:21.102570 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:21.102707 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:21.108801 (+ 6094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 02:42:21.108808 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 02:42:21.108813 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 02:42:21.108852 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 02:42:21.116460 (+ 7608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:21.118020 (+ 1560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:21.118108 (+ 88us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:21.118112 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:21.118381 (+ 269us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 02:42:21.119112 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 02:42:21.119118 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 02:42:21.119122 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 02:42:21.119968 (+ 846us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 02:42:21.121190 (+ 1222us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:21.122182 (+ 992us) server_negotiation.cc:299] Negotiation successful
0812 02:42:21.123692 (+ 1510us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260812 02:42:21.147565 2945 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 866606d3e56d426586c079860654745d: Bootstrap starting.
I20260812 02:42:21.154510 2945 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 866606d3e56d426586c079860654745d: Neither blocks nor log segments found. Creating new log.
I20260812 02:42:21.156684 2945 log.cc:824] T 00000000000000000000000000000000 P 866606d3e56d426586c079860654745d: Log is configured to *not* fsync() on all Append() calls
I20260812 02:42:21.157243 3076 sys_catalog.cc:455] T 00000000000000000000000000000000 P c430004289ea4516bc94ad66d9f69cb3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader da8eed8aa8f2485ea71b535405643c34. Latest consensus state: current_term: 1 leader_uuid: "da8eed8aa8f2485ea71b535405643c34" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da8eed8aa8f2485ea71b535405643c34" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 43121 } } peers { permanent_uuid: "866606d3e56d426586c079860654745d" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 44363 } } peers { permanent_uuid: "c430004289ea4516bc94ad66d9f69cb3" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 34455 } } }
I20260812 02:42:21.157568 3076 sys_catalog.cc:458] T 00000000000000000000000000000000 P c430004289ea4516bc94ad66d9f69cb3 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 02:42:21.169517 2945 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 866606d3e56d426586c079860654745d: No bootstrap required, opened a new log
I20260812 02:42:21.187867 2945 raft_consensus.cc:348] T 00000000000000000000000000000000 P 866606d3e56d426586c079860654745d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da8eed8aa8f2485ea71b535405643c34" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 43121 } } peers { permanent_uuid: "866606d3e56d426586c079860654745d" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 44363 } } peers { permanent_uuid: "c430004289ea4516bc94ad66d9f69cb3" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 34455 } }
I20260812 02:42:21.188715 2945 raft_consensus.cc:729] T 00000000000000000000000000000000 P 866606d3e56d426586c079860654745d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 866606d3e56d426586c079860654745d, State: Initialized, Role: FOLLOWER
I20260812 02:42:21.189752 2945 consensus_queue.cc:260] T 00000000000000000000000000000000 P 866606d3e56d426586c079860654745d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da8eed8aa8f2485ea71b535405643c34" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 43121 } } peers { permanent_uuid: "866606d3e56d426586c079860654745d" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 44363 } } peers { permanent_uuid: "c430004289ea4516bc94ad66d9f69cb3" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 34455 } }
I20260812 02:42:21.194412 3054 sys_catalog.cc:455] T 00000000000000000000000000000000 P da8eed8aa8f2485ea71b535405643c34 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "da8eed8aa8f2485ea71b535405643c34" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da8eed8aa8f2485ea71b535405643c34" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 43121 } } peers { permanent_uuid: "866606d3e56d426586c079860654745d" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 44363 } } peers { permanent_uuid: "c430004289ea4516bc94ad66d9f69cb3" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 34455 } } }
I20260812 02:42:21.194757 3054 sys_catalog.cc:458] T 00000000000000000000000000000000 P da8eed8aa8f2485ea71b535405643c34 [sys.catalog]: This master's current role is: LEADER
I20260812 02:42:21.195303 3088 catalog_manager.cc:1347] Generated new cluster ID: f52ad464c5ef4f5fbabd37da8c14a75b
I20260812 02:42:21.195423 3088 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 02:42:21.201449 2916 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "da8eed8aa8f2485ea71b535405643c34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "866606d3e56d426586c079860654745d"
I20260812 02:42:21.201822 2915 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 866606d3e56d426586c079860654745d [term 0 FOLLOWER]: Advancing to term 1
I20260812 02:42:21.202229 2945 sys_catalog.cc:564] T 00000000000000000000000000000000 P 866606d3e56d426586c079860654745d [sys.catalog]: configured and running, proceeding with master startup.
I20260812 02:42:21.205621 2915 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 866606d3e56d426586c079860654745d [term 1 FOLLOWER]: Refusing update from remote peer da8eed8aa8f2485ea71b535405643c34: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 02:42:21.207195 3076 sys_catalog.cc:455] T 00000000000000000000000000000000 P c430004289ea4516bc94ad66d9f69cb3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "da8eed8aa8f2485ea71b535405643c34" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da8eed8aa8f2485ea71b535405643c34" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 43121 } } peers { permanent_uuid: "866606d3e56d426586c079860654745d" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 44363 } } peers { permanent_uuid: "c430004289ea4516bc94ad66d9f69cb3" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 34455 } } }
I20260812 02:42:21.207473 3076 sys_catalog.cc:458] T 00000000000000000000000000000000 P c430004289ea4516bc94ad66d9f69cb3 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 02:42:21.206835 3081 consensus_queue.cc:1035] T 00000000000000000000000000000000 P da8eed8aa8f2485ea71b535405643c34 [LEADER]: Connected to new peer: Peer: permanent_uuid: "866606d3e56d426586c079860654745d" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 44363 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260812 02:42:21.209599 2916 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 866606d3e56d426586c079860654745d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da8eed8aa8f2485ea71b535405643c34 in term 1.
I20260812 02:42:21.211287 3107 sys_catalog.cc:455] T 00000000000000000000000000000000 P 866606d3e56d426586c079860654745d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da8eed8aa8f2485ea71b535405643c34" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 43121 } } peers { permanent_uuid: "866606d3e56d426586c079860654745d" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 44363 } } peers { permanent_uuid: "c430004289ea4516bc94ad66d9f69cb3" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 34455 } } }
I20260812 02:42:21.211795 3107 sys_catalog.cc:458] T 00000000000000000000000000000000 P 866606d3e56d426586c079860654745d [sys.catalog]: This master's current role is: FOLLOWER
I20260812 02:42:21.230221 3109 catalog_manager.cc:1259] Loaded cluster ID: f52ad464c5ef4f5fbabd37da8c14a75b
I20260812 02:42:21.233129 3088 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 02:42:21.233995 3109 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c430004289ea4516bc94ad66d9f69cb3: loading cluster ID for follower catalog manager: success
I20260812 02:42:21.235353 3088 catalog_manager.cc:1504] Loading token signing keys...
W20260812 02:42:21.235451 3109 catalog_manager.cc:1576] T 00000000000000000000000000000000 P c430004289ea4516bc94ad66d9f69cb3: acquiring CA information for follower catalog manager: Not found: root CA entry not found
W20260812 02:42:21.235587 3109 catalog_manager.cc:873] Not found: root CA entry not found: failed to prepare follower catalog manager, will retry
I20260812 02:42:21.259285 3107 sys_catalog.cc:455] T 00000000000000000000000000000000 P 866606d3e56d426586c079860654745d [sys.catalog]: SysCatalogTable state changed. Reason: New leader da8eed8aa8f2485ea71b535405643c34. Latest consensus state: current_term: 1 leader_uuid: "da8eed8aa8f2485ea71b535405643c34" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da8eed8aa8f2485ea71b535405643c34" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 43121 } } peers { permanent_uuid: "866606d3e56d426586c079860654745d" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 44363 } } peers { permanent_uuid: "c430004289ea4516bc94ad66d9f69cb3" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 34455 } } }
I20260812 02:42:21.259636 3107 sys_catalog.cc:458] T 00000000000000000000000000000000 P 866606d3e56d426586c079860654745d [sys.catalog]: This master's current role is: FOLLOWER
I20260812 02:42:21.259945 3117 sys_catalog.cc:455] T 00000000000000000000000000000000 P 866606d3e56d426586c079860654745d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "da8eed8aa8f2485ea71b535405643c34" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da8eed8aa8f2485ea71b535405643c34" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 43121 } } peers { permanent_uuid: "866606d3e56d426586c079860654745d" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 44363 } } peers { permanent_uuid: "c430004289ea4516bc94ad66d9f69cb3" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 34455 } } }
I20260812 02:42:21.260427 3117 sys_catalog.cc:458] T 00000000000000000000000000000000 P 866606d3e56d426586c079860654745d [sys.catalog]: This master's current role is: FOLLOWER
I20260812 02:42:21.230332 2956 master_runner.cc:186] Error getting master registration for 127.7.132.189:44363: OK, instance_id { permanent_uuid: "866606d3e56d426586c079860654745d" instance_seqno: 1786502540365260 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 02:42:21.264205 2956 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 02:42:21.278172 3124 sys_catalog.cc:455] T 00000000000000000000000000000000 P da8eed8aa8f2485ea71b535405643c34 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "da8eed8aa8f2485ea71b535405643c34" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da8eed8aa8f2485ea71b535405643c34" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 43121 } } peers { permanent_uuid: "866606d3e56d426586c079860654745d" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 44363 } } peers { permanent_uuid: "c430004289ea4516bc94ad66d9f69cb3" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 34455 } } }
I20260812 02:42:21.278812 3124 sys_catalog.cc:458] T 00000000000000000000000000000000 P da8eed8aa8f2485ea71b535405643c34 [sys.catalog]: This master's current role is: LEADER
I20260812 02:42:21.283488 3088 catalog_manager.cc:5868] T 00000000000000000000000000000000 P da8eed8aa8f2485ea71b535405643c34: Generated new TSK 0
I20260812 02:42:21.285091 3088 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 02:42:21.317567 3132 catalog_manager.cc:1259] Loaded cluster ID: f52ad464c5ef4f5fbabd37da8c14a75b
I20260812 02:42:21.317746 3132 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 866606d3e56d426586c079860654745d: loading cluster ID for follower catalog manager: success
I20260812 02:42:21.321501 3132 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 866606d3e56d426586c079860654745d: acquiring CA information for follower catalog manager: success
I20260812 02:42:21.324323 3132 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 866606d3e56d426586c079860654745d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260812 02:42:21.406618 7698 mini_kdc.cc:348] Time spent kdestroy: real 0.105s user 0.000s sys 0.031s
W20260812 02:42:21.453341 3050 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:42:21.444116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37512 (local address 127.7.132.190:43121)
0812 02:42:21.444355 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:21.444361 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:21.449771 (+ 5410us) server_negotiation.cc:407] Connection header received
0812 02:42:21.451571 (+ 1800us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:21.451578 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:21.451672 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:21.451752 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:21.453057 (+ 1305us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37512: BlockingRecv error: recv got EOF from 127.0.0.1:37512 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
W20260812 02:42:21.455771 2951 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:42:21.440276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39262 (local address 127.7.132.189:44363)
0812 02:42:21.440510 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:21.440517 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:21.449214 (+ 8697us) server_negotiation.cc:407] Connection header received
0812 02:42:21.454012 (+ 4798us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:21.454019 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:21.454158 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:21.454248 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:21.455492 (+ 1244us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39262: BlockingRecv error: recv got EOF from 127.0.0.1:39262 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
W20260812 02:42:21.460310 3052 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:42:21.450554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51158 (local address 127.7.132.188:34455)
0812 02:42:21.450743 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:21.450750 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:21.457825 (+ 7075us) server_negotiation.cc:407] Connection header received
0812 02:42:21.458822 (+ 997us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:21.458829 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:21.458930 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:21.459008 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:21.460038 (+ 1030us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51158: BlockingRecv error: recv got EOF from 127.0.0.1:51158 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
I20260812 02:42:21.521308 2951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:21.498412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39270 (local address 127.7.132.189:44363)
0812 02:42:21.498626 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:21.498633 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:21.511750 (+ 13117us) server_negotiation.cc:407] Connection header received
0812 02:42:21.512426 (+ 676us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:21.512434 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:21.512661 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:21.512810 (+ 149us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 02:42:21.514239 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:21.515511 (+ 1272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:21.518061 (+ 2550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:21.518753 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:21.518869 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:21.519677 (+ 808us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 02:42:21.520442 (+ 765us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 02:42:21.520580 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:21.520841 (+ 261us) server_negotiation.cc:299] Negotiation successful
0812 02:42:21.520963 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
W20260812 02:42:21.566071 3052 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:42:21.555753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51170 (local address 127.7.132.188:34455)
0812 02:42:21.555969 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:21.555975 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:21.563780 (+ 7805us) server_negotiation.cc:407] Connection header received
0812 02:42:21.564377 (+ 597us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:21.564384 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:21.564541 (+ 157us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:21.564624 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:21.565797 (+ 1173us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51170: BlockingRecv error: recv got EOF from 127.0.0.1:51170 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
I20260812 02:42:21.612805 2951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:21.597514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39284 (local address 127.7.132.189:44363)
0812 02:42:21.597712 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:21.597719 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:21.605242 (+ 7523us) server_negotiation.cc:407] Connection header received
0812 02:42:21.606188 (+ 946us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:21.606195 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:21.606312 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:21.606394 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 02:42:21.607095 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:21.608305 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:21.610418 (+ 2113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:21.611139 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:21.611236 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:21.611898 (+ 662us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 02:42:21.612058 (+ 160us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 02:42:21.612129 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:21.612365 (+ 236us) server_negotiation.cc:299] Negotiation successful
0812 02:42:21.612467 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
W20260812 02:42:21.659731 3052 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:42:21.649464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51172 (local address 127.7.132.188:34455)
0812 02:42:21.649695 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:21.649702 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:21.657539 (+ 7837us) server_negotiation.cc:407] Connection header received
0812 02:42:21.658068 (+ 529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:21.658119 (+ 51us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:21.658228 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:21.658314 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:21.659459 (+ 1145us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51172: BlockingRecv error: recv got EOF from 127.0.0.1:51172 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
I20260812 02:42:21.711982 2951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:21.694595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39300 (local address 127.7.132.189:44363)
0812 02:42:21.694781 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:21.694788 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:21.706756 (+ 11968us) server_negotiation.cc:407] Connection header received
0812 02:42:21.707019 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:21.707032 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:21.707109 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:21.707243 (+ 134us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 02:42:21.707746 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:21.708669 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:21.710066 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:21.710522 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:21.710668 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:21.711121 (+ 453us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 02:42:21.711250 (+ 129us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 02:42:21.711364 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:21.711516 (+ 152us) server_negotiation.cc:299] Negotiation successful
0812 02:42:21.711602 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
W20260812 02:42:21.731014 3052 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:42:21.728428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51182 (local address 127.7.132.188:34455)
0812 02:42:21.728659 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:21.728668 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:21.729243 (+ 575us) server_negotiation.cc:407] Connection header received
0812 02:42:21.729486 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:21.729492 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:21.729575 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:21.729715 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:21.730740 (+ 1025us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51182: BlockingRecv error: recv got EOF from 127.0.0.1:51182 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
I20260812 02:42:21.744753 2951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:21.738507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39304 (local address 127.7.132.189:44363)
0812 02:42:21.738774 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:21.738781 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:21.739116 (+ 335us) server_negotiation.cc:407] Connection header received
0812 02:42:21.739399 (+ 283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:21.739404 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:21.739482 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:21.739605 (+ 123us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 02:42:21.740059 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:21.741101 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:21.742811 (+ 1710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:21.743283 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:21.743432 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:21.743868 (+ 436us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 02:42:21.744034 (+ 166us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 02:42:21.744139 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:21.744302 (+ 163us) server_negotiation.cc:299] Negotiation successful
0812 02:42:21.744411 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113}
W20260812 02:42:21.753111 3052 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:42:21.751056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51186 (local address 127.7.132.188:34455)
0812 02:42:21.751341 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:21.751348 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:21.751651 (+ 303us) server_negotiation.cc:407] Connection header received
0812 02:42:21.751872 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:21.751878 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:21.751967 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:21.752076 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:21.752894 (+ 818us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51186: BlockingRecv error: recv got EOF from 127.0.0.1:51186 (error 108)
Metrics: {"server-negotiator.queue_time_us":130}
I20260812 02:42:21.771577 2951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:21.765654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39316 (local address 127.7.132.189:44363)
0812 02:42:21.765899 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:21.765923 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:21.766584 (+ 661us) server_negotiation.cc:407] Connection header received
0812 02:42:21.766834 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:21.766839 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:21.766916 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:21.767026 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 02:42:21.767479 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:21.768226 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:21.769569 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:21.770042 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:21.770259 (+ 217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:21.770712 (+ 453us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 02:42:21.770825 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 02:42:21.770940 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:21.771127 (+ 187us) server_negotiation.cc:299] Negotiation successful
0812 02:42:21.771216 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
W20260812 02:42:21.782449 3052 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:42:21.780199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51188 (local address 127.7.132.188:34455)
0812 02:42:21.780409 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:21.780416 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:21.780648 (+ 232us) server_negotiation.cc:407] Connection header received
0812 02:42:21.780927 (+ 279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:21.780932 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:21.781031 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:21.781152 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:21.782167 (+ 1015us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51188: BlockingRecv error: recv got EOF from 127.0.0.1:51188 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
I20260812 02:42:21.808952 2951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:21.802854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39322 (local address 127.7.132.189:44363)
0812 02:42:21.803094 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:21.803105 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:21.803720 (+ 615us) server_negotiation.cc:407] Connection header received
0812 02:42:21.803965 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:21.803972 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:21.804064 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:21.804177 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 02:42:21.804623 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:21.805452 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:21.807069 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:21.807534 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:21.807687 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:21.808149 (+ 462us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 02:42:21.808272 (+ 123us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 02:42:21.808380 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:21.808536 (+ 156us) server_negotiation.cc:299] Negotiation successful
0812 02:42:21.808633 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
W20260812 02:42:21.820298 3052 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:42:21.817821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51194 (local address 127.7.132.188:34455)
0812 02:42:21.818109 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:21.818118 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:21.818703 (+ 585us) server_negotiation.cc:407] Connection header received
0812 02:42:21.818938 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:21.818943 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:21.819026 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:21.819146 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:21.819966 (+ 820us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51194: BlockingRecv error: recv got EOF from 127.0.0.1:51194 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
I20260812 02:42:21.862223 2951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:21.856165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39332 (local address 127.7.132.189:44363)
0812 02:42:21.856403 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:21.856413 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:21.857008 (+ 595us) server_negotiation.cc:407] Connection header received
0812 02:42:21.857362 (+ 354us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:21.857369 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:21.857480 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:21.857537 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 02:42:21.858109 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:21.858857 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:21.860222 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:21.860705 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:21.860850 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:21.861269 (+ 419us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 02:42:21.861401 (+ 132us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 02:42:21.861505 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:21.861667 (+ 162us) server_negotiation.cc:299] Negotiation successful
0812 02:42:21.861755 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
W20260812 02:42:21.872231 3052 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:42:21.868747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51202 (local address 127.7.132.188:34455)
0812 02:42:21.868998 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:21.869007 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:21.870648 (+ 1641us) server_negotiation.cc:407] Connection header received
0812 02:42:21.870871 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:21.870878 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:21.870977 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:21.871122 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:21.871971 (+ 849us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51202: BlockingRecv error: recv got EOF from 127.0.0.1:51202 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
I20260812 02:42:21.945823 2951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:21.939714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39348 (local address 127.7.132.189:44363)
0812 02:42:21.939944 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:21.939954 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:21.940543 (+ 589us) server_negotiation.cc:407] Connection header received
0812 02:42:21.940789 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:21.940794 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:21.940867 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:21.940981 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 02:42:21.941442 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:21.942319 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:21.943685 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:21.944203 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:21.944363 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:21.944922 (+ 559us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 02:42:21.945033 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 02:42:21.945146 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:21.945311 (+ 165us) server_negotiation.cc:299] Negotiation successful
0812 02:42:21.945399 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
W20260812 02:42:21.953989 3052 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:42:21.951985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51218 (local address 127.7.132.188:34455)
0812 02:42:21.952164 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:21.952171 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:21.952578 (+ 407us) server_negotiation.cc:407] Connection header received
0812 02:42:21.952728 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:21.952734 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:21.952844 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:21.952960 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:21.953756 (+ 796us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51218: BlockingRecv error: recv got EOF from 127.0.0.1:51218 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 02:42:22.093333 2951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:22.087975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39358 (local address 127.7.132.189:44363)
0812 02:42:22.088166 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:22.088171 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:22.088511 (+ 340us) server_negotiation.cc:407] Connection header received
0812 02:42:22.088678 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:22.088682 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:22.088756 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:22.088850 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 02:42:22.089290 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:22.090041 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:22.091444 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:22.091905 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:22.092002 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:22.092568 (+ 566us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 02:42:22.092684 (+ 116us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 02:42:22.092800 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:22.092959 (+ 159us) server_negotiation.cc:299] Negotiation successful
0812 02:42:22.093043 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
W20260812 02:42:22.101500 3052 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 02:42:22.099458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51230 (local address 127.7.132.188:34455)
0812 02:42:22.099705 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:22.099713 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:22.100103 (+ 390us) server_negotiation.cc:407] Connection header received
0812 02:42:22.100327 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:22.100332 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:22.100407 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:22.100525 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0812 02:42:22.101280 (+ 755us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51230: BlockingRecv error: recv got EOF from 127.0.0.1:51230 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 02:42:22.239192 3109 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c430004289ea4516bc94ad66d9f69cb3: acquiring CA information for follower catalog manager: success
I20260812 02:42:22.241211 3109 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c430004289ea4516bc94ad66d9f69cb3: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 02:42:22.367071 2951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:22.361617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39362 (local address 127.7.132.189:44363)
0812 02:42:22.361949 (+ 332us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:22.361957 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:22.362112 (+ 155us) server_negotiation.cc:407] Connection header received
0812 02:42:22.362422 (+ 310us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:22.362426 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:22.362505 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:22.362614 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 02:42:22.363073 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:22.363865 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:22.365283 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:22.365700 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:22.365843 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:22.366284 (+ 441us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 02:42:22.366425 (+ 141us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 02:42:22.366531 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:22.366695 (+ 164us) server_negotiation.cc:299] Negotiation successful
0812 02:42:22.366781 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105}
I20260812 02:42:22.379423 3052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:22.372953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51232 (local address 127.7.132.188:34455)
0812 02:42:22.373142 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:22.373147 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:22.373643 (+ 496us) server_negotiation.cc:407] Connection header received
0812 02:42:22.373860 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:22.373866 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:22.374051 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:22.374243 (+ 192us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 02:42:22.374648 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:22.375408 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:22.376950 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:22.377426 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:22.377612 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:22.378210 (+ 598us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 02:42:22.378781 (+ 571us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 02:42:22.378964 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:22.379029 (+ 65us) server_negotiation.cc:299] Negotiation successful
0812 02:42:22.379134 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 02:42:22.393622 2951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:22.385626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39376 (local address 127.7.132.189:44363)
0812 02:42:22.385828 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:22.385836 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:22.386901 (+ 1065us) server_negotiation.cc:407] Connection header received
0812 02:42:22.387206 (+ 305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:22.387212 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:22.387306 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:22.387378 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 02:42:22.388361 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:22.389313 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:22.391448 (+ 2135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:22.391890 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:22.392037 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:22.392734 (+ 697us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 02:42:22.392877 (+ 143us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 02:42:22.392963 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:22.393176 (+ 213us) server_negotiation.cc:299] Negotiation successful
0812 02:42:22.393262 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 02:42:22.394147 3052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:22.386962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51244 (local address 127.7.132.188:34455)
0812 02:42:22.387201 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:22.387210 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:22.388343 (+ 1133us) server_negotiation.cc:407] Connection header received
0812 02:42:22.388574 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:22.388580 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:22.388659 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:22.388758 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 02:42:22.389313 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:22.390070 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:22.391998 (+ 1928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:22.392724 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:22.392853 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:22.393324 (+ 471us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 02:42:22.393442 (+ 118us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 02:42:22.393548 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:22.393742 (+ 194us) server_negotiation.cc:299] Negotiation successful
0812 02:42:22.393832 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260812 02:42:22.394526 3271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 02:42:22.385524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37516 (local address 127.7.132.190:43121)
0812 02:42:22.385998 (+ 474us) server_negotiation.cc:206] Beginning negotiation
0812 02:42:22.386002 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 02:42:22.386224 (+ 222us) server_negotiation.cc:407] Connection header received
0812 02:42:22.386521 (+ 297us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 02:42:22.386529 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 02:42:22.386796 (+ 267us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 02:42:22.386988 (+ 192us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 02:42:22.387813 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:22.388910 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:22.391231 (+ 2321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 02:42:22.391879 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 02:42:22.392407 (+ 528us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 02:42:22.392892 (+ 485us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 02:42:22.393458 (+ 566us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 02:42:22.394031 (+ 573us) server_negotiation.cc:1035] Waiting for connection context
0812 02:42:22.394114 (+ 83us) server_negotiation.cc:299] Negotiation successful
0812 02:42:22.394207 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":310,"threads_started":1}
I20260812 02:42:22.402328 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 2774
I20260812 02:42:22.403621 2849 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:42:22.517706 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 2774
W20260812 02:42:22.545061 2887 connection.cc:535] server connection from 127.0.0.1:39218 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 02:42:22.545559 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 2866
I20260812 02:42:22.546802 2939 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:42:22.657155 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 2866
I20260812 02:42:22.683913 7698 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu pid 2956
I20260812 02:42:22.685209 3038 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 02:42:22.799243 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMpwqb/build/asan/bin/kudu with pid 2956
2026-08-12T02:42:22Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (5089 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (10529 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (385799 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST